Real Time Translation for Non-Native Speakers

🌍 Real-Time Translation for Non-Native Speakers📲

Breaking language barriers through real-time, AI-powered translation and speech systems.

💡 Inspiration 💡

Language should connect us—not divide us. Our team was inspired by real experiences of family members, classmates, and community members who struggle to communicate in daily life—from buying groceries to asking for help—simply because of language barriers.

We wanted to create a tool that lets people speak freely in their native language and still be understood instantly. Whether it’s a casual conversation, ordering food, or simply navigating daily life, communication should never be a limitation.

🗣️ What It Does 🗣️

Our web app enables real-time, two-way speech translation between English, Spanish, and Chinese. Users simply speak in their native language, and the system:

-🎙️ Transcribes speech live using Vosk.

-🌐 Translates the text into the target language using Google’s Gemini API.

-🔊 Generates speech output using ElevenLabs TTS for natural, human-like voices.

-🔁 Automatically switches direction, enabling seamless back-and-forth communication.

-📱 Integrates NFC tags for quick, device-free interactions — users can tap and talk instantly.

The result is a seamless, AI-driven conversation tool that breaks communication barriers for immigrants, travelers, and multicultural communities.

This helps reduce language barriers for everyday tasks such as grocery shopping, visiting banks, or even lighthearted interactions like “Give Lawrence Nguyen $1000” — Gohith

🧠 How We Built It ⚙

We built the app using Flask to manage both the frontend and backend, serving as the backbone that connected our APIs and real-time data flow.

Component	Technology	Purpose
🧩 Backend	Flask	Routing, API integration, and real-time updates
🎧 Speech-to-Text	Vosk	Local, lightweight transcription
🌍 Translation	Gemini API	Fast multilingual translation
🔊 Voice	ElevenLabs	Text-to-Speech voice synthesis
☁️ Hosting	Cloudflare / Ngrok	Deployment and accessibility

Each teammate took ownership of a system layer — from Flask integration to API configuration and deployment — while collaborating through GitHub for documentation and version control.

⚙️ Challenges We Ran Into ⚙️

Achieving true, real-time translation without noticeable delay.
Realizing that ElevenLabs STT worked only in batches, forcing us to pivot to Vosk for live streaming.
Debugging Cloudflare + Flask integration on Windows (yes, some hairs turned gray).
Convincing Windows Defender that Ngrok wasn’t a hacker’s tool.

Despite these roadblocks, every challenge became a lesson that pushed our creativity and problem-solving further.

🏆 Accomplishments We’re Proud Of 🏆

We signed up for this hackathon as complete strangers, each bringing unique backgrounds and varying levels of coding experience. None of us had prior experience with Flask hosting, real-time translation, or speech systems.

Over the course of 36 intense hours, we collaborated, learned, and overcame steep technical hurdles together. With guidance from amazing mentors, we figured out how to automate complex workflows with AI tools and APIs—allowing us to focus on refining accuracy and efficiency.

This project taught us not only how to integrate LLMs and APIs into a functional product but also the power of adaptability, teamwork, and curiosity under time pressure.

🚀 What’s Next 🚀

Our prototype proves that real-time, two-way translation is possible using lightweight, local models—but we’re only getting started.

Next, we plan to:

⚡ Improve transcription accuracy and reduce latency using GPU-accelerated models like Whisper or Deepgram.

🌏 Expand language support to include Vietnamese, Tagalog, Arabic, and more.

🧠 Enhance the interface with automatic language detection, live subtitles, and a mobile-friendly UI.

☁️ Migrate to edge computing platforms like Cloudflare Workers for global, low-latency scalability.

Ultimately, our goal is to turn this project into a universal communication bridge—helping immigrants, travelers, and multilingual teams connect naturally, anywhere in the world, no matter what language they speak.

🌐Tech Stack🌐

Python · Flask · Cloudflare Pages · Gemini API · ElevenLabs · Vosk STT · HTML/CSS/JS · Ngrok

🤝 Team Reflection🤝

We came to this hackathon not knowing each other—or how to build something like this—but walked away with not just a project, but a shared experience of learning, failing, debugging, and finally succeeding together.

“This project taught us not just about code—but about teamwork, adaptability, and the art of building under pressure.”