Seeing how most people love to dance 🕺 to their favourite/popular music (in TikTok 😋, on the dancefloor 💃, etc.), we wanted to create something that further allows everyone to be part of the song they dance to. Let your freak out 🤘, or carry out some simple choreography, and let our portable device—JikJukBox—open up a new world 🌎 of music-making for you! 🎼🎵
What it does
JukJikBox contains a motion sensor that captures sensor data when you move to your favourite beats. You start off with your favourite pop song from your device while an audio visualizer plays (we have projected it out in our demo video). With the JukJikBox strapped to your arm, you begin dancing to the music. As you show off your sick moves and dance skills, your favourite music changes in audio amplitude/volume and pitch, somewhat like a DJ track. You jump and twist some more, and you hear tambourines and random sound effects playing in the background. Amazed, you loudly think, "Interesting.. Let me keep dancing!"
How we built it
We used an ESP32 microcontroller to collect acceleration and gyroscope data from a MPU-6050 sensor. With the use of socket.io, the sensor data is reliably transferred over WiFi with low latency to a web server. The web server then processes the sensor data to transform the music according to user’s movements with P5.js and Tone.js libraries.
Challenges we ran into
- High latency with POST request from ESP32 to remote server
- Not able to set up socket.io on ESP32
- Configure and calibrate the MPU-6050 sensor
- Find a suitable JS library to manipulate the sound and generate music
Accomplishments that we're proud of
- Realtime changes to music from user movements (see video of sensor data collection here)
What we've learnt
- Socket.io, client and server
- I2C to interface with sensor
- P5.js and Tone.js to process the music
- Handicraft when building the box
What’s next for JikJukBox
- Allow for more sounds and types of audio manipulation for your favourite songs (to dance!)
- Make JikJukBox famous on TikTok
Log in or sign up for Devpost to join the conversation.