Aeva: The AI Reality Companion for True Fluency

Build native-mode thinking by immersing in the small. Let AI explain everything in front of you: in real life, in any virtual place, or in any video.

Comment

Another youtube link https://youtu.be/a59-zC4y-7U

Inspiration

Traditional language learning is a failure of method—the "Mute English" trap is a deep, long-term pain for immigrants and students who can’t form a "native mindset" even after years abroad. We knew that true immersion is the only cure, but it’s an expensive luxury—requiring native families, schools, or partners that most can't access. We built Aeva to be the mentor we never had: a multimodal companion that shares your view. During development, I felt a literal "hit"—it’s addictive to point your camera at a sidewalk curb, a specific pillar, or a Chipotle menu and have an AI instantly teach you the names of the world in real-time. We moved from "dead" passive input like Netflix (where you watch for 8 hours with zero engagement) to a high-stakes, closed-loop feedback engine that turns walking into a high-intensity learning adventure of discovery.

What it does

Aeva is architected on two orthogonal axes. Axis one is High-Intensity Pedagogy: specifically designed modes like English-to-English vocabulary mapping (explaining objects with basic words to kill the translation habit), "Build-Up" listening (short to long sentence drills), and authentic paraphrasing. Axis two is Contextual Versatility: we’ve turned the entire planet into a classroom. You can use your camera for real-world AR, voice-control your navigation through Google Street View ("Take me to Wall Street," "Turn left"), or co-watch any YouTube story with synchronized context. We’ve even integrated a "Persistent Soul" via local Markdown, Supabase, and Notion, ensuring Aeva remembers every mistake, every level (A1-C2), and every preference across sessions.

How we built it

We evolved Aeva from a simple multimodal prototype into a resilient, production-grade ecosystem. We focused on "Aliveness"—engineering a custom Node.js WebSocket proxy on Google Cloud Run to keep the audio/video stream secure and ultra-low latency. We scaled the vision from local camera feeds to global explorations and infinite YouTube stories. Behind the scenes, we maintained professional rigor: CI/CD-driven staging/production separation, AI-assisted code reviews via "vibe coding," and robust retry logic. We learned to balance the speed of AI iteration with the stability required for a real-world learning tool.

Challenges we ran into

Mastering "Vibe Coding" at scale was our steepest climb. We learned that you can't just trust the AI; you need a human-in-the-loop to handle the "merge friction" when combining features from different models. We faced brutal regressions during integration—especially with new frontends and tools—which we fought off with product-level "double-checking" and a growing suite of unit tests. On the media front, we conquered audio concurrency wars in YouTube mode and fixed the notorious Street View "black screen" bug by optimizing our rate-limiting and filtering out broken photosphere data. We learned that the faster the iteration, the more rigorous the verification must be.

Accomplishments that we're proud of

We are most proud of finding the path to "Immersion for Everyone." We turned the painful, passive chore of language study into a dopamine-driven, active exploration. We achieved a synergy where $1+1+1 > 3$—combining Gemini’s real-time multimodal intelligence with autonomous "Auto-Chat" persistence, voice-navigated global street-walks, and shared media context. Watching the AI naturally "teleport" through the world or co-watch a video while holding a persistent memory of the user’s "Soul" is a breakthrough that gives me immense satisfaction. This isn't just a tool; it's a living reality companion.

What we learned

This project proved that Voice is the definitive interface for the next generation of computing. We learned that while AI can implement the "Shell" of a product with incredible speed, the "Soul" and direction must remain human-driven—rooted in a founder’s obsession and personal pain. We also gained a deep respect for the friction of AI-driven collaboration; managing the divergence of different LLMs requires professional infrastructure and a clear, dominant product vision. True excellence happens when you bridge the gap between AI intelligence and a massive, personal human problem.

What's next for Aeva

Aeva is moving from the browser to the pocket. Our top priority is a Native Mobile App for true, high-mobility outdoor immersion. We are diving deeper into the science of linguistics and neurology to further refine our "Build-Up" and "Shadowing" techniques. We’re also expanding the "Soul" layer—making the personalization so deep that Aeva knows your history, your mistakes (mapped via memory.md), and your personal goals better than your own mirror. We’re building toward a world where you inhabit the language you learn.

Built With

Share this project:

Updates

Submission history