A deep and unreasonable love of xylophones
What it does
An air xylophone right in your browser! Play such classic songs as twinkle twinkle little star, ba ba rainbow sheep and the alphabet song or come up with the next club banger in free play.
We also added an air guitar mode where you can play any classic 4 chord song such as Wonderwall
How we built it
We built a static website using React which utilised Posenet from TensorflowJS to track the users hand positions and translate these to specific xylophone keys.
We then extended this by creating Xylophone Hero, a fun game that lets you play your favourite tunes without requiring any physical instruments.
Challenges we ran into
Fine tuning the machine learning model to provide a good balance of speed and accuracy
Accomplishments that we're proud of
I can get 100% on Never Gonna Give You Up on XylophoneHero (I've practised since the video)
What we learned
We learnt about fine tuning neural nets to achieve maximum performance for real time rendering in the browser.
What's next for XylophoneHero
We would like to:
- Add further instruments including a
guitarand drum set in both freeplay and hero modes
- Allow for dynamic tuning of Posenet based on individual hardware configurations
- Add new and exciting songs to Xylophone
- Add a multiplayer jam mode