Kpop Demon Hunters Academy

Inspiration

Our biggest inspiration for this project was Kpop Demon Hunters (also, obviously the greatest movie released in 2025). After hearing about how Caleb and Skyler passionately got their high school friends to cosplay the Saja boys in theaters to watch the movie—for the fifth time—we were energized. The music, choreography, and energy sparked an idea: what if we could actually train like them?

What it does

Kpop Demon Hunters Academy merges the capabilities of Just Dance with that of a karaoke application, allowing users to sing, move, and level up their idol skills in real time with our trainer. To use it:

Upload any Kpop mp4 video with a dancer, and the app extracts the song's pitch and beat. If you don't have, we have options to choose from!
As you sing, dance, or both, the website tracks your pitch, rhythm, and body to give you a real-time accuracy score and help you improve your skills. It's that simple!

How we built it

We first began by splitting into two components: dancing and karaoke.

Two of us began working on the dancing component, first prototyping the product as a website that, given an uploaded video file, could track human limbs in frame and visualize them back to the user. We used the JavaScript build tool Vite for the frontend tooling, and utilized Google's open-source media streaming framework MediaPipe for the computer vision task. We were also able to connect to the user's webcam and stream it. We then noticed that the user's video would take too long to process after uploading, so we switched to the Lite model and changed the conference parameters until processing was almost immediate. At this point, one of us improved and tested the dancer scoring algorithm, while the other designed the UI and UX in KPop Demon Hunters style.

Simultaneously, the other two of us first focused on getting two sound inputs, one from the uploaded mp4 file, and another from the user's input. One tool that we found specifically useful was Web Audio API, which helped us with audio processing, pitch detection, user microphone capture, audio visualization, and scoring. By referencing and improving upon pre-existing karaoke machine models, we were able to find a balance of weights between pitch accuracy (50-70%), timing accuracy (30%), and vocal stability (20%). For scoring, we used a zone-based averaging model with confidence weighting.

Once both of these products were complete, we merged them onto a single website that could achieve both tasks with the same video.

Challenges we ran into

One of our biggest challenges was finding a way to combine both of our functionalities into one cohesive experience. As mentioned earlier, we built our karaoke and dance components separately. Although we tried to maintain modular, reusable code structures and consistent data flow, merging them into a single app while ensuring the UI worked seamlessly for both meant making tough calls about which features to keep, simplify, or remove. Having an amazing and memorable user experience was the most important step for us.

Another challenge was the lack of online information there was on real-time pitch and rhythm analysis using the Web Audio API, especially when integrating it with YouTube-based audio sources. We had to experiment with different approaches to pitch detection, latency correction, and buffer synchronization to ensure it worked smoothly across a variety of audio samples.

Finally, time management was a major hurdle. Integrating audio processing, visual feedback, and animation while preparing for our pitch presentations under hackathon conditions pushed our limits. Despite these obstacles, seeing everything come together was incredibly rewarding.

What we learned

Beyond the countless technical skills this hackathon experience enabled us to build, we learned a lot about the importance of debugging and writing clean, friendly code. Our codebase wasn't something we had to write and pass along without consequence; on the contrary, having to merge our work with our teammates meant we were uniquely compelled to ensure that our code was readable and understandable, not only by us but by others as well. Doing this across such a vast project took a lot more time than we anticipated, and we learned valuable skills in formatting, good coding practices, and resilience.

What's next for Kpop Demon Hunters Academy

We have BIG PLANS for Kpop Demon Hunters Academy in the future. We hope to add multiplayer modes so that users can battle or duet with friends in real time. There would also be a singleplayer mode where users dance alongside AI dancers. Other ideas include creating an AI-based feedback system to help users improve vocal/dancing technique, designing a mobile companion app, and hosting global leaderboards and seasonal events. The sky's the limit!