Lyrica

Landing Page

🎤 You already know this song. Can you sing it in Korean?

Lyrica teaches you a language by having you sing it. Karaoke apps score your notes. Lyrica also listens for your words.

Inspiration

The three of us come from different countries, and we all learned languages the same way: through music. Before we were comfortable speaking English, we were singing it. Songs gave us the words and the confidence to say them out loud.

But no app is built around that. Language apps teach you to read and tap. Singing apps only check your notes. So we built the one we wish we'd had.

What it does

Pick a language. Happy Birthday in English, Korean, or Arabic. You know the tune, so you only learn the words.
Hear it. An ElevenLabs voice speaks the line.
Sing it. Your pitch draws live against the target notes.
Get two scores.
- Tune: did you hit the notes?
- Words: did you sing the words, or just hum?
Meet Aux. Our AI ref calls the round and tells you one thing to try next.
Play solo or battle a friend, best of three, with a global leaderboard.

Try this: hum the tune. Tune stays high, Words drops to almost nothing. Then sing it. That gap is the whole idea.

How we built it

Mic → Browser (live pitch) → Tune score
            ↓
        Backend (FastAPI)
        ├─ Speech model (PyTorch) → Words score
        ├─ Gemini → Aux's feedback
        ├─ ElevenLabs → Aux's voice + line audio
        └─ Tiger Data → leaderboard

Design: We designed the interface in Figma and used Figma Make to generate the first working prototype, then rebuilt the logic ourselves in React.

Frontend: React, TypeScript, Vite, Tailwind. Pitch detection runs in the browser with the Web Audio API.

Tune score (our own code): we compare the shape of your melody to the target, measured from your own middle note. A deep voice and a high voice get the same score for the same performance. An automated test proves it: a take shifted down a full octave scores identically.

Words score (machine learning): a pre-trained wav2vec 2.0 speech model in PyTorch compares your take to a reference clip. We recorded 18 test takes to check it. All 12 takes with the right words scored above all 5 takes with no words (humming, "la la la," mumbling, whistling, noise).

Best Use of ElevenLabs

Every line in all three languages is spoken by an ElevenLabs voice, so learners hear correct pronunciation before they sing.
Aux has one consistent ElevenLabs voice, generated live from each round's feedback.

Best Use of Gemini

Gemini writes Aux's feedback from the real round data: who took it, what each player did well, one thing to try next.
Gemini never sees or changes a score. Our code decides the winner, then tells Gemini.
Aux is kind by design. Learning a language out loud is vulnerable, so Aux never comments on a voice or accent and never says what you got wrong.

Best Use of Tiger Data

Every turn is saved as a time-stamped row in a Tiger Data hypertable.
That powers the leaderboard and a "hardest syllables" view that only time-series turn data can answer.
Players opt in. We store a nickname and scores, never audio.

Challenges we ran into

Unsingable targets. Our first melody jumped almost two octaves in one word. Nobody could score above 50.
Fairness. Comparing raw notes meant deeper voices always lost. Scoring the shape fixed it.
Honest ML. Our speech model reliably separates words from humming. Telling right words from wrong words needs more data, so we ship what it does well and say where it stops.
Songs. We couldn't use popular songs, so we chose a tune the whole world already knows.