Here at Pixelhop, things have been getting spooky, our bones have been rattling away, and we've been busy experimenting with some new technology and building out something fun and scary. Do you dare find out more?
In October we've built a mini experience for Halloween centred around using pose detection to map a skeleton onto a users body in real-time! We had great fun putting it together and animating the design, and we think it is a great example of how powerful browsers have become! You can try it out here: Trick or Treat
It's only a tiny microsite, but it packs in lots of cool tech and APIs, including:
Pose detection with TensorFlow.js and the WebGL backend
The MediaRecorder API to record users transformations
The Web Share API allows users to share their reactions in other apps.
Web Audio API for sound effects
Vue 3 and Vite
Bucket loads of CSS animations
We will be writing a series of blog posts going through the design, animations and more of the technical side of things, explaining exactly how we tackled the project. If you're interested in these, then subscribe or follow us on Twitter (@pixelhopio) to get updates.
In the meantime, here's a bit more background on how we started the project.
What's the point?
One thing we've always loved about our jobs is working on experimental projects with new technologies. It's not often you get to do in work hours, and it falls to having to do in our evenings and weekend as side projects. It's something we're beginning to prioritise at Pixelhop, especially as the team starts to grow. Now that our new developer Talie has settled in and Halloween was approaching, we thought it was perfect timing to book our Trick or Treat project.
We had a workshop in the office and tried to thrash out some ideas of what, whys and hows. While we work remotely, it's nice to get together now and then, particularly when throwing ideas around. During these sessions, we don't have any tech; we just use good old fashioned pens and paper. We find it opens up an opportunity to really think, come up with ideas out of the box and just open up our creative mind palaces.
We start all projects like this and make sure we answer the why, whats, and hows to keep us focused.
Why?
The why was mainly to have fun and learn something new in the process. We also see experiments like these as a marketing opportunity to show potential clients the sort of creative things we can make.
What?
We then spoke about some of the new pieces of tech we wanted to play with and what would be good to create a little Halloween mini experience that people might want to play with and share. We decided TensorFlow pose detection would be fun to try out. It's cutting edge technology, and we could really have some fun mapping a skeleton to someone's body.
How?
We then just needed to define precisely how we were going to do it. We had already decided to use TensorFlow.js for the pose detection, and for the rest of the build we decided to go for our preferred stack of Vue 3, Vite, and Tailwind CSS.
When?
If we could have, we would have spent all month on it, but we had a fixed deadline so we reduced the scope to something we knew we would be able to produce in time. We had 3 days for design and 2 weeks to develop and write the blog posts on how it was built. There are undoubtably areas we would like to improve if we had more time such as making the trick or treat paths more unique. At the moment the only thing that changes is the skull mapped to your body. There are also some performance issues that appear when the pose detection starts that can be quite jarring, but overall we are pretty happy with how polished the project turned out.
Happy Halloween!
We hope you enjoyed playing with a Halloween web experiment. We would love to know what you think and if there is any way you would improve it? Likewise, if you have any questions for us, give us a shout on Twitter: @pixelhopio or get in contact via email.
Zef and I are constantly talking about different ideas and "things" we want to build and projects we want to start. Over the years we have started countless side projects but have finished only a few. It's hard to keep side project momentum around busy lifestyles and full-time jobs.
