Metamatik

Metamatik teaches young kids foundational math through mixed-reality games played with hands, face, and body — guided by Lumi, a voice AI tutor that reacts in real time.

Comment

What inspired Metamatik

This started at our kitchen table. Senem teaches math, and for years she's studied how young children build a sense of number long before they memorize "2 + 2." Kids who develop number sense early — recognizing quantities at a glance, comparing magnitudes, feeling how numbers relate — find later math far less intimidating. But most apps drill symbols and right answers, not the intuition underneath. We wanted to build what we wished existed for our own child: math as play, explored with the hands, face, and body, where a kid never feels tested.

What it is

Metamatik is a mixed-reality math library for children roughly 4–8. It's a curriculum of topics and subtopics, and every subtopic is a short game the child plays through their camera with hand, face, and body gestures instead of tapping buttons. Number Sense is fully built — thirteen games across four sessions, from recognizing and counting quantities to number relationships, early operations, and estimation — and Geometry, Patterns, and Measurement add more playable games, with the subtopics still to come shown as locked cards.

A voice AI tutor named Lumi, a small glowing firefly, stays with the child throughout. Lumi follows each game in real time — it knows the current activity and what the child is doing — so it guides gently, gives feedback in the moment, and answers questions in plain language, like a patient tutor. The camera is read entirely on the device: only gestures reach the game, never the video.

For example, in Guess the Swarm a swarm of fireflies flashes on screen and the child estimates how many, then puffs their cheeks to lock the guess in. Every game stays playable with touch or a mouse when the camera, microphone, or voice service isn't available, and progress is saved on the device.

How we built it

Senem owns the pedagogy — the curriculum, the learning goal behind every game, and checking each against how children actually develop number sense. Bedirhan (a medical doctor working as a software engineer) built the platform. It runs on Next.js, React, and Cloudflare Workers; gestures are recognized in the browser with MediaPipe (hand, face, and pose), games render as 2D canvas scenes, and Lumi speaks through the OpenAI Realtime API over WebRTC, with a browser-speech fallback.

On the OpenAI side, GPT-5.6 powers Lumi's reasoning, its awareness of the game state and the child's actions, and its in-the-moment feedback. We used Luna for rapid prototyping, escalated to Sol with max reasoning on a hard problem, and leaned on GPT-5.6 for brainstorming and deep research. Codex was our build partner throughout — scaffolding the app and iterating on game mechanics — letting a two-person team move like a bigger one.

What we learned

The hardest part of early-math edtech isn't the tech — it's restraint. A young child doesn't need more features; they need fewer, clearer moments of "I get it." Pairing a teacher's judgment with a model that adapts in real time let us cut, not add.

Challenges

Getting Lumi to act like a warm tutor rather than a quiz engine took real iteration — when to stay quiet, when to nudge, how to phrase feedback for a child. Making gesture interaction reliable across ordinary webcams, while keeping the camera on-device for privacy, was the other big one.

What's next

Unlocking the subtopics still shown as locked cards, deepening Geometry, Patterns, and Measurement until each is as complete as Number Sense, and testing with real classrooms to see what actually sticks.

Built With

Share this project:

Updates