LinguaScene

Inspiration

Language barriers create serious problems in everyday situations, from healthcare and travel to work and simple day-to-day communication. Hundreds of millions of people live in places where they are not fully fluent in the dominant language, and even learners who study for years often struggle when they need to actually speak in real time. Most language learning tools focus on vocabulary and grammar in isolation, but they don’t prepare you for real conversations where you have to understand, respond, and think quickly under pressure. That gap between learning a language and actually using it in real situations is what LinguaScene was built to close.

What it does

LinguaScene drops you into immersive, real-life conversational scenarios like a doctor's consultation in Spanish, a café order in French, a job interview in Mandarin and has you converse with an AI character to complete a goal. The AI responds naturally in the target language, corrects mistakes in context with inline tips, and scores your full session at the end with a grammar breakdown, strengths, and one concrete improvement tip. Voice input and output are fully supported, so you can speak and listen rather than type.

How we built it

• Frontend: React 18 + Vite + TailwindCSS, deployed on Vultr.
• Backend: FastAPI (Python 3.11) deployed on Railway, exposing two endpoints: /chat for conversation turns and /score for end-of-session evaluation.
• AI: We used the Gemini API to power the character roleplay, inline corrections, and structured session scoring. Each call includes the full conversation history so the AI maintains character and context across all 8 turns.
• Voice: Web Speech API handles the speech recognition (input in the target language).
• Scenarios: Nine pre-built scenes, each with a character, setting, goal, and vocabulary targets. Also the functionality to build your own scenes, customized to your unique requirements.

Challenges we ran into

• Voice API quirks: Web Speech API is Chrome-only and behaves differently across operating systems. Getting the language codes to switch correctly between all five languages (especially Mandarin with zh-CN) and loading voices asynchronously without race conditions took significant debugging.
• Keeping the AI in character: Prompt engineering to make the AI maintain its persona while still delivering natural corrections and tips, without breaking immersion, required many iterations. We settled on a [💡 tip: ...] delimiter convention that let us parse and display tips separately without speaking them aloud.
• Stateless AI with stateful conversation: Gemini has no memory between calls, so we pass the full conversation history on every request. Managing that payload cleanly and keeping latency under 3 seconds per turn was a careful balancing act.

Accomplishments that we're proud of

• A fully deployed, publicly accessible web app built and shipped in 48 hours by a team of four.
• End-to-end voice I/O working reliably in a live demo environment.
• A healthcare-focused scenario (Doctor Consultation) that uses real medical vocabulary, realistic symptom-description goals, and meaningful AI feedback.

What we learned

Building LinguaScene showed us that learning a language and using a language are very different things. Users don’t just need knowledge, they need practice responding in real time under realistic conditions. Designing conversation-based scenes made that especially clear. A café, an airport, or a market each creates a different kind of language use, and engagement increases when the setting feels familiar and practical. We also learned that prompt engineering is a product skill, not just a technical one. The difference between an AI that feels like a helpful tutor and one that feels like a robotic translator often comes down to how the system prompt is written.

What's next for LinguaScene

• Improving User accounts and session history so learners can track improvement over time.
• Native mobile app (React Native) for on-the-go practice.
• Expanded language and scenario library — 5+ new scenes, more languages, and a gamification layer (streaks, XP) to drive daily engagement.

Built With

• axios
• gemini
• javascript
• python
• react
• tailwindcss
• uvicorn
• vite
• vultr
• webspeechapi