MunchinBuddy

Inspiration

One in five children in the U.S. is obese, and childhood Type 2 diabetes diagnoses have surged 95% in the last decade. We noticed a gap: kids don't engage with nutrition data the way adults do — calorie counts and macro tables mean nothing to a 7-year-old. And pediatricians often rely on brief office visits with no data between appointments. What if healthy eating felt like a game, your food coach was a buddy who talks back to you, and your doctor got AI-generated clinical reports backed by weeks of real data? That spark became MunchinBuddy — an AI-powered virtual character that physically morphs based on what a child eats, turning every meal into instant visual feedback a kid can understand, while building a data trail that helps families and doctors intervene early.

What it does

For Kids 🎮

• Voice-first meal logging — Kids simply say "I just had two eggs and toast" and the AI parses calories, macros, and micronutrients automatically.
• Morphing avatar — The buddy flexes muscles after protein, gets sluggish and sick after junk food, and glows with energy after balanced meals — immediate, visceral feedback.
• Smart intent detection — "I want a milkshake" gets nutrition advice; "I had a milkshake" logs the meal. The AI understands the difference.
• Gamification — Streaks, XP, levels, badges, and a school leaderboard make healthy eating competitive and fun.
• Continuous voice conversation — Like ChatGPT voice mode — the AI speaks back in a kid-friendly voice (ElevenLabs TTS), then the mic auto-restarts for natural back-and-forth.

For Parents 👨‍👩‍👧

• Parent dashboard — Detailed daily stats, eaten-vs-recommended nutrient bars across all macros and micronutrients.
• Blood report analysis — Upload lab reports (CBC, metabolic panels). The AI extracts values, compares across multiple reports over time, and correlates improvements with actual diet and activity data.
• Apple Watch integration — Activity data (steps, calories burned, exercise minutes) feeds into AI recommendations.

For Pediatricians 🩺

• AI-generated clinical reports — One-click weekly/monthly reports designed for pediatrician review, including:
  • Patient overview (age, weight, activity level)
  • Energy balance analysis (intake vs. burn) with daily trend charts
  • Macro split visualization and daily averages table
  • Micronutrient adequacy bars against age-appropriate RDAs
  • Blood report flag summary with longitudinal comparison
  • AI narrative highlighting concerns, improvements, and discussion points
• Pre-computed daily stats — Average calories, protein, iron, Vitamin D, sodium, fiber, steps, exercise minutes — all stored per day and instantly queryable for any date range.
• Blood report correlation — The AI connects lab value changes to actual dietary patterns: "Average iron intake of 8mg/day is below the 10-15mg recommendation, explaining persistent low ferritin despite improvement from 8 → 12 ng/mL." ## How we built it | Layer | Tech | |---|---| | Frontend | Next.js 16 (App Router), React 19, Tailwind CSS v4, Recharts | | AI Engine | AWS Nova Lite via Bedrock — intent classification, meal parsing, nutrition coaching, report generation, blood report extraction | | Voice | Web Speech API (recognition) + ElevenLabs streaming TTS | | Database | SQLite + Prisma ORM with pre-computed DailyStat aggregations | | Activity Data | Simulated Apple Watch data (realistic scenario-based generation) |

Cost-optimized AI architecture

A key design decision was choosing AWS Nova Lite over more expensive models like Claude or GPT-4. We achieved comparable output quality at a fraction of the cost through context engineering:

• Structured system prompts — Instead of relying on the model's general knowledge, we feed pre-computed stats, nutrient targets, and blood report diffs directly into tightly scoped prompts. The model summarizes and correlates rather than reasons from scratch.
• Mode-aware prompt switching — A single endpoint serves kid, parent, and pediatrician audiences by swapping system prompts, not models. The kid prompt enforces 3-4 excited sentences; the parent prompt demands specific numbers; the pediatrician prompt produces clinical markdown.
• Router agent pattern — A lightweight classifier (just 40 output tokens) routes every message to log_meal, ask_nutrition, or general_chat, each with specialized prompts. This keeps per-request context small and focused.
• Pre-aggregated data — Daily stats are computed and stored in the database on every meal log. When the AI generates a report or answers a question, it receives a clean summary table — not thousands of raw meal records. This keeps prompt tokens minimal.
• Deterministic parsing — Meal parsing uses strict JSON output schemas, reducing hallucination and retry costs. This architecture means scaling to thousands of users doesn't require scaling to expensive models. Nova Lite at ~$0.06/1M input tokens handles our workload at roughly 1/50th the cost of frontier models, with quality maintained by giving the model better context, not more parameters. ## Challenges we faced
• Intent ambiguity — "I want pizza" vs "I had pizza" required careful prompt engineering and fallback keyword logic to avoid logging meals the child hasn't eaten yet.
• Voice-before-text synchronization — The AI's text response appeared before the voice finished loading. We solved this with an onStart callback from the audio player that reveals the text bubble only when speech actually begins.
• Stale avatar state — The avatar was stuck in "strong" mode after junk food because it used a rolling average. We pivoted to a last-meal-only state model persisted in the database.
• Blood-to-diet correlation — Connecting lab value changes to dietary patterns required building a daily stats pipeline and crafting prompts that teach the AI to draw meaningful clinical connections without hallucinating causation.
• Making cheap models smart — Nova Lite out-of-the-box produced generic responses. The breakthrough was realizing that better context beats bigger models — feeding it pre-computed averages, RDA comparisons, and blood report diffs made its output clinically useful. ## What we learned
• Kids need immediate, visual, emotional feedback — not numbers. The avatar morphing after junk food is 10x more impactful than showing "720 kcal, 28g sat fat."
• Voice is the natural interface for children. Typing is a barrier; talking is play.
• Context engineering > model size. A $0.06/M-token model with the right data in the prompt outperforms a $15/M-token model given raw data and vague instructions.
• Pre-computing and storing daily aggregates transforms a chatbot into an analyst that can reference weeks of data in milliseconds.
• Designing for two audiences (playful kid vs. clinical pediatrician) in one system requires mode-switching at the prompt level, not the model level. ## What's next
• Real Apple Watch / HealthKit integration via iOS companion app
• OCR + Vision API for grocery receipt scanning and meal photo recognition
• Pediatrician portal with shareable report links and patient management
• FHIR integration for direct EHR data exchange
• Multi-language support for broader accessibility
• School district pilot program with anonymized cohort analytics

Built With

• ai
• nextjs