landing page

Brain Lens

"The AI tutor that sees, speaks, and guides you to the solution."

Inspiration

We’ve all been there: staring at a math problem late at night, stuck on a single step. You don't need the final answer—you need a nudge in the right direction. Existing tools often just spit out the solution (like $x = 42$), which robs the student of the "Aha!" moment.

We were inspired to build Brain Lens to bridge the gap between a static answer key and a real human tutor. We wanted an AI that could see your messy handwriting, listen to your confusion, and talk you through the logic step-by-step, just like a teacher sitting next to you.

What it does

Brain Lens is a real-time, multimodal AI tutor.

Vision: You point your camera at a math or science problem.
Voice: You talk to it naturally. "I'm stuck on this integral."
Guidance: It responds with voice and text, never giving the answer directly, but guiding you.

For example, if you are solving $\int x^2 dx$, and you guess "Is it $2x$?", Brain Lens won't just say "No." It might say, "That would be the derivative. Remember the power rule for integration adds to the exponent?"

How we built it

We built Brain Lens using a modern web stack designed for real-time interaction:

Next.js & TypeScript: For a responsive, robust frontend application.
Google Gemini 2.0 Flash (Multimodal Live API): The core intelligence. We utilize Gemini's native capability to process video frames and audio streams simultaneously with low latency.
Web Audio API: To capturing microphone input and stream it to the model.
HTML5 Canvas: To capture video frames from the user's camera, optimize them, and send them to the vision model.
Vercel: For seamless deployment.

The "Tutor" Persona

A key part of the build was the system instruction (prompt engineering). We strictly instructed the model:

"Never give the full answer directly. Guide the student step by step. Celebrate small wins."

Challenges we faced

Real-time Latency: Achieving the "conversational" feel was tough. We had to optimize how often we sent video frames to Gemini to balance bandwidth with responsiveness.
Audio Feedback Loops: Since the AI speaks while the user might be speaking, handling audio interruptions and preventing the AI from hearing itself was a significant hurdle.
Hallucinations: Sometimes the AI would try to solve the problem too quickly. We had to refine the prompts to ensure it waits for the user's input before jumping ahead.

What we learned

Building Brain Lens taught us the immense power of Multimodal AI. It’s not just about text anymore; it’s about context. When the AI can see what the user sees, the interaction becomes infinitely more grounded and helpful.

We also learned that pedagogy is hard. Teaching isn't just knowing facts; it's knowing when to reveal a fact and when to hold back. Encoding that intuition into software was our biggest takeaway.