Inspiration
Inspired by the movie Beauty and the Beast, we set out to do something similar. In the movie, there are multiple iconic scenes in which everyday household objects, such as a teapot and a candlestick, come to life and burst into song. We tried to capture the magic of watching the movie for the first time by creating Band of the Cursed, in which the instruments on the screen come to life depending on You!
What it does
The four instruments on the screen light up individually when a sound from their instrument is played which is detected using sound classification. There is also an AR option which allows you to view the instruments in AR.
How we built it
We used ml5 to create the sound classification models and used Krita to animate and draw the instruments. We also used Three.js to create the AR environemnt. The sound classification models were trained using individual layers of Led Zeppelin's Stairway to Heaven.
Challenges we ran into
Learning how to animate in Krita was hard
Accomplishments that we're proud of
Learning how to animate objects and using sound classification
What's next for Band of the Cursed
Make the AI more accurate, especially with other things playing in the background.
Built With
- ml5
- three.js
Log in or sign up for Devpost to join the conversation.