Pratibha AI

A voice-first AI tutor for children who lack access to a teacher. Speaks their language. Sees through the camera. Remembers every child. No typing. No English. Just talk.

Comment

The Story Behind Pratibha AI

What Inspired Me

My mother's name is Pratibha. She was a teacher in a small town in India. Every day she would go to school, stand in front of 40 children, and try her best to reach every single one of them. She couldn't always. There were too many children, too few hours, too little support.

That memory stayed with me.

When I look at AI today, I see something that troubles me. The most powerful technology ever built is reaching the people who need it least. Developers in San Francisco. Students at top universities. Professionals who already have every advantage.

The child sitting in a rural classroom in Bihar, whose teacher didn't show up today — she has no access to any of this. Not because the technology doesn't exist. Because nobody built it for her.

AI is not for everyone. Not yet. Pratibha is my attempt to change that.

What I Built

Pratibha is a real-time voice-first AI tutor for children who lack access to a teacher. A child speaks naturally in Hindi or any regional language. An animated teacher avatar listens, responds with voice, generates educational illustrations, and identifies real-world objects through the camera. No typing. No reading. No English required.

The camera does not open when the child presses a button. It opens when Pratibha asks to see. That single design decision defines the whole product — it feels like a real teacher, not a tool.

How I Built It

Built solo in 48 hours.

Frontend: React + Vite, mobile-first portrait layout
AI: Gemini Live 2.5 Flash via Vertex AI — real-time bidirectional audio and vision over WebSocket
Image Generation: Gemini 2.5 via Vertex AI — triggered automatically when Pratibha explains a concept
Database: Google Cloud Firestore — stores each child's name, class, and weak topics persistently
Backend: Node.js WebSocket proxy — handles service account auth so credentials never touch the client
Audio: Web Audio API with a PCM (Pulse Code Modulation) AudioWorklet capturing 16kHz mono — the exact format Gemini Live expects
Camera: getUserMedia with adaptive quality — automatically adjusts frame resolution and send rate based on network speed using navigator.connection API
Deployment: Google Cloud Run

The avatar is built from 4 static .webp frames swapped programmatically tied to actual audio playback state — lighter than any video stream and gives precise lip-sync control.

What I Learned

WebSocket proxying is non-negotiable for browser security. Browsers cannot attach a service account token directly to a WebSocket connection. The Node.js proxy handles authentication transparently.

Tool calling requires careful silence management. When Pratibha saves a child's profile silently, the AI must not speak. Suppressing tool acknowledgment from triggering a new audio turn took careful state management.

Camera context must reset on every session. Gemini retains visual memory across turns. Without an explicit reset instruction, Pratibha would hallucinate objects from a previous camera session.

The hardest problem was not technical. It was making the experience feel human. An AI that waits to be asked before opening the camera feels like a teacher. An AI with a button feels like a tool. That difference is everything.

Challenges

  • Getting raw 16kHz PCM audio from the browser without MediaRecorder (which outputs webm containers Gemini cannot consume) required building a custom AudioWorklet processor
  • Avatar animation flickering caused by background images during crossfade — solved by converting all frames to .webp with transparent backgrounds and using z-index swapping instead of opacity transitions
  • Double audio playback when tools were called — Gemini generates audio before and after tool execution, requiring a tool processing flag to suppress the first response
  • Image generation rate limiting (HTTP 429) under load — solved with a simple 2-second retry
  • Making the experience work on 2G connections — adaptive camera compression and .webp avatar frames kept the UI responsive even on slow networks

The Name

Pratibha means talent and brilliance in Sanskrit. It is also my mother's name.

She spent her life trying to reach every child in her classroom. This is my way of helping her finally do that.

Built With

Share this project:

Updates

Submission history