Lingua Talk: Talk, Listen and Learn

Tired of learning a language for years but still can't understand or speak it? Why not try Lingua Talk? There's no need to type—just talk and listen as if you're conversing with a native speaker.

Comment

Inspiration

From developer's own language learning experience and pain points

What it does

Use your voice to interact with a curated chatbot powered by the OpenAI API. Ask questions, listen to responses in real-time, and learn any language effortlessly.

How we built it

Collaborate with a team of engineers, machine learning engineers, and data scientists to quickly build an API and web app with token funding from OpenAI Research Group

Challenges we ran into

We have never built a full stack app before. It is a great learning experience

  • Developing a proof of concept (POC) in a notebook is easy, but creating a web app is more complex. Beyond learning backend APIs and setting up databases, we faced challenges ensuring smooth communication between frontend and backend with minimal latency. Initially, we limited GPT responses, but caching audio files in the browser and directly triggering endpoints improved user experience with minimal lag.

  • Cost: Deploying on AWS ECS was convenient but costly, with $10 spent over one weekend of minimal traffic. We switched to Railway.app’s $5/month plan, successfully hosting the web app at a sustainable cost.

  • CI/CD: After local development and Docker testing, we deployed the app on Railway. Manual updates became impractical, so we integrated GitHub for automatic FastAPI updates with each push, streamlining continuous deployment.

Accomplishments that we're proud of

  • Developed an Innovative Voice Interaction System which helps people particularly community of language learners do not have access to native speakers
  • Build out first working app with my friends during our spare time
  • Optimized Deployment and Cost Efficiency

What we learned

API design, CICD and AWS cloud service and railway.app for hosting web app

What's next for Voice Controlled Cooking Assistant

The app serves as a platform to apply our research in LLMs, such as fine-tuning models for specific languages or audience groups, to provide a better learning experience. We can use school syllabuses as a reference to help people learn in a more structured, step-by-step manner.

Built With

Share this project:

Updates