Peekaboo

Your family photos become your child's first vocabulary lesson.

Comment

Inspiration

This idea started from watching my own kid point at things and ask what they were called, growing up in a multilingual environment. Other vocabulary apps rely on generic, non-personalized photos and pictures. There's no real connection for the child, so they feel distant from the material and forget it easily. The bicycle in our park, the toy on the floor, breakfast this morning: those are the words that stick. That's the problem Peekaboo solves.

What it does

Peekaboo turns private family photos into mobile, audio-first French or English vocabulary games.

A parent uploads a photo, and Peekaboo identifies visible, concrete objects and proposes age-appropriate vocabulary, pronunciations, and playful teaching lines. Before anything reaches the child, the parent reviews the suggestions, changes ambiguous labels, and removes unsuitable objects.

The child then explores the original photo in Child Mode. They tap an approved object, hear its name and a short teaching phrase, repeat the word using optional speech recognition, and celebrate learning it. Progress is saved independently for each child and language track, with lightweight review scheduling that brings words back over time.

People can appear naturally in family photos, but they are never playable and are never made to "talk."

How we built it

Peekaboo is a mobile-first React and TypeScript app with an Express API and shared Zod contracts. The approach was to stay as close to vanilla as possible and only reach for a framework or library once we actually hit a wall, rather than defaulting to a heavy stack upfront.

The pipeline uses a configurable vision-language model for vocabulary and safety metadata, Grounding DINO for object localization, and SAM 2 segmentation when available, with a rectangle-based fallback when it isn't. Browser speech synthesis handles audio, and forgiving speech recognition lets kids try saying a word without ever blocking their progress.

Family photos stay private: anonymous Supabase auth, row-level security, private storage, short-lived signed URLs, server-side credentials. Only parent-approved, non-person objects become playable. A frontend-only demo mode keeps photos entirely in-browser, no external credentials needed.

It's deployed on Render for both frontend and backend, with Supabase as the database, and relies on third-party APIs like OpenRouter and Replicate for the AI processing.

Codex was an engineering partner throughout: turning the concept into a production-ready monorepo, working through the privacy architecture, building the vision pipeline and bilingual learning tracks, diagnosing detection bugs, and prepping deployment. I made the core product and safety decisions; Codex helped implement and verify them, with GPT-5.6 running inside Codex during development.

Challenges we ran into

The hardest part wasn't the idea. It was making AI vision behave predictably enough to trust around a child. Early on, detection would return malformed boxes or point at the wrong region of the photo, a real problem when a five-year-old taps the screen expecting the word to match what they touched.

The design leans on a simple principle: AI proposes, a parent decides. Every detection is schema-validated, low-confidence results are filtered by default, rectangles cover for segmentation when it's unavailable, and nothing reaches the child without parent approval first.

Accomplishments that we're proud of

I'm proud that this isn't just a vision demo I could show off once. It's something I'd actually hand to my own kid: take a photo, review it, hand over the phone, watch them learn a word from their own life.

What we learned

I learned that AI vision output needs to be treated as a starting point, not a final answer, especially when children are the end user. Trustworthy AI features aren't about the model being right more often, they're about designing for what happens when it's wrong.

What's next for Peekaboo

Peekaboo could help multilingual and language-learning families make practice more meaningful, without asking children to adapt to another generic curriculum. Their home, travels, toys, and memories become the curriculum. Next steps would be expanding beyond French and English, and building out the spaced-review system so words genuinely stick over weeks, not just one session.

Built With

Share this project:

Updates