SingBack

Inspiration

This project was inspired by The Choicer Voicer, a voice acting game that's been going viral on social media lately. We loved its simple recording and reveal format, and wanted to bring that same chaotic, social fun to singing and musical memory.

What it does

SingBack is a local multiplayer party game that puts your musical memory to the test. Players hear a short song clip once, then take turns singing it back over the instrumental. Once everyone finishes, the performances are revealed and scored on pitch, rhythm, lyrics, and completion, with personalized AI feedback for each player.

How we built it

We built SingBack with React, TypeScript, and Vite on the frontend, and FastAPI on the backend. Browser audio is captured with MediaRecorder, mixed with instrumentals using FFmpeg, and analyzed with Librosa for pitch and rhythm. ElevenLabs handles lyric transcription and host narration, while Gemini turns the measured results into personalized feedback. We also used AI coding tools to help with brainstorming, scaffolding, and debugging. Our team made the product decisions, integrated the systems, and tested the complete game flow.

Challenges we ran into

One of our biggest challenges was making the grading feel fair. ElevenLabs sometimes missed or cut out lyrics from short, noisy recordings, so we had to handle incomplete transcriptions without breaking the score. We also experimented with how much pitch, rhythm, lyrics, and completion should matter before settling on a 45/25/20/10 split. Along the way, we fixed cases where different vocal ranges or missing audio could produce misleading results.

Performance was another major issue. Pitch detection and waveform generation were initially slow because we reanalyzed the original song for every player. We sped this up by caching its audio features, processing performances in parallel, and reducing unnecessary waveform detail (10,000 to 180 bins). We also added timeouts and local fallbacks so slow AI services would never trap players on the loading screen.

Accomplishments that we're proud of

We’re proud that we built a complete multiplayer game in one weekend, since at one point we were worried we weren't going to finish in time. We also made the scoring explainable by showing pitch traces, waveforms, lyric accuracy, and clear metric breakdowns instead of relying on a mysterious AI-generated number. Furthermore, SingBack still works when Gemini or ElevenLabs is slow or unavailable, making the experience both fun and reliable.

What we learned

We learned that browser audio is much harder than simply pressing record. Small differences in microphone volume, playback timing, and vocal range can completely change the results, so good preprocessing and testing mattered just as much as the scoring algorithm. We also learned that AI works best as a supporting layer during ideation and revision. Oftentimes the AI wanted to implement features that were unnecessary or overengineered. Finally, we learned that ideation takes much longer than we thought, and that getting started in time matters more than finding the perfect idea.

What's next for SingBack

Next, we want to expand SingBack with more song clips pulled from a licensed music API, multiplayer support across separate devices, and a practice mode that tracks how your singing improves over multiple attempts. We’d also love to add more game modes and make SingBack something an entire room can play together. Eventually, we'd like to support online multiplayer and Omegle-style drop-in rooms where strangers can compete together.