Lingo

Dashboard
Word Stress sorting game
Speech emphasis game
Listen and interpret emphasis game
Speech statistics

Inspiration

One of our teammates is an ESL teacher at W&L's ESOL program, a nonprofit serving underserved Hispanic populations. Her experience highlighted a need that can persist even among advanced learners: practicing the stress and rhythm of spoken English.

We created Lingo to make these patterns approachable through short, engaging games.

What it does

Lingo explores spoken English through five mini-games:

Stress Sort: Categorize words by which syllable is stressed.
Stress Twins: Which word keeps the same stress as a noun and a verb?
What Do They Mean?: Listen and interpret how sentence emphasis shapes meaning.
Rhythm Match: Listen and identify matching beat patterns.
Say it Right: Speak sentences with correct word stress.

Our goal is to support confident communication while respecting learners’ accents and identities.

How we built it

We organized the experience around three linguistic topics: word stress, sentence stress, and speech rhythm. We translated these concepts into interactive challenges with a simple interface inspired by daily puzzle games.

Our teammate’s teaching experience helped guide the learning goals. As we refined the project, we reduced overlapping activities and gave each game a distinct purpose.

Infrastructure:

Domain: lingo.study (free via GoDaddy)
Hosting: DigitalOcean App Platform

Stack:

Backend: FastAPI + Uvicorn + Python
Frontend: React + TypeScript + Vite + Auth0
AI/Speech: ElevenLabs API, Web Speech API, Media Recorder API

Challenges we faced

Our biggest design challenge was balancing linguistic depth with simplicity. Several early games tested similar skills, so we consolidated them into four focused activities.

We also had to distinguish what the games actually measure. Recognizing a stress pattern or interpreting emphasis does not automatically demonstrate speaking fluency. That distinction helped us frame feedback around performance on the activities.

What we learned

We learned how closely sound, grammar, and meaning interact—and how challenging it is to turn those relationships into clear learning experiences. Also, adding more games does not necessarily create more educational value: each activity needs a specific purpose and understandable feedback. Most importantly, we discovered that accessible, engaging education can significantly impact underserved learners' confidence and communication opportunities.

What’s next

We want to test Lingo with ESL learners and teachers, expand the examples, and refine the audio activities. Future development could include a mobile app with push notifications, facial recognition for mouth tracking during speech, and more advanced speech feedback evaluation.

Built With

css
digitalocean
ffmpeg
lucide
next.js
python
radix
react
recharts
tailwind
typescript
ui

Submitted to

&HACKS XII
- Winner Silver Prizes
- Winner [MLH] Best Use of ElevenLabs
- Winner Linguistics

Created by

I worked across product design and frontend. I designed how the linguistic concepts would translate into game mechanics and the overall user experience, and then helped build those designs into the actual interface.

Paula Yaniz
I built the app's audio and speaking features on ElevenLabs. I created a pipeline that generates stress-accurate model recordings, using SSML phoneme tags for word stress, v3 emphasis for sentence stress, and automatic measurement to pick the best take, plus timestamps for synced word highlighting. I designed and built Say It Right, a speaking game that uses ElevenLabs Speech-to-Text and a stress checker I wrote and validated to tell players whether they stressed the right words.

Devin Upadhyay
I worked on the back end.

Danilo Šindrić
Anh Nhu Ha Xuan