Caliwo (Ambient Nutrition Intelligence Hub)

Speak Your Meals. Master Your Goals

Comment

Inspiration

Dietary accountability is universally acknowledged as the single most critical factor in preventative health, yet over 80% of people abandon nutrition tracking within their first three weeks. The reason isn't lack of willpower—it is interaction friction.

Standing in the kitchen with your hands covered in olive oil or holding a cooking spatula while trying to unlock a phone, search a database with 50,000 entries, weigh ingredients, and manually type grams into a mobile app breaks the natural flow of daily life.

We asked ourselves: Why are we treating nutrition tracking like a spreadsheet data-entry job when the kitchen is already home to ambient smart speakers and displays?

With the emergence of Amazon Bedrock and the Model Context Protocol (MCP), we realized we could completely reimagine nutrition tracking. Instead of forcing users onto smartphone screens, Caliwo turns dietary accountability into an ambient, hands-free conversation that lives natively in your kitchen.

What it does

Caliwo is an ambient, voice-first AI nutrition intelligence agent designed for the next-generation Amazon Alexa+ ecosystem. It bridges voice assistants and personal health data into a unified, real-time feedback loop:

  1. Proactive Nutritional Gap Analysis: When a user asks, "Alexa, what should I eat tonight for dinner?", Caliwo doesn’t just read a random recipe off the web. Its autonomous agent inspects the user’s live intake against personalized targets, identifies precise nutritional deficiencies (e.g., “You are 35g behind on protein with 520 calories remaining”), and suggests a tailored meal that perfectly closes that gap.

  2. Context-Aware One-Shot Logging ("Add That"): Users never recite numbers or macro splits. Simply saying "Add that", "Lock it in", or "I just ate three scrambled eggs and whole wheat toast" prompts the agent to dynamically calculate portions, estimate calories and macronutrients on the fly, and invoke the log_meal tool to record the entry.

  3. Simulated Alexa+ Hardware Experience (assistant/): Judges and users can interact with a dedicated smart display simulator featuring glowing ambient Alexa light rings, responsive audio spectrum waveforms, natural neural voice synthesis, and real-time visual macro HUD cards.

  4. Transparent MCP Tool Telemetry: As the agent reasons, interactive telemetry badges pop up live on-screen (e.g., [get_nutrition_gaps], [log_meal], [get_daily_nutrition]), providing visible, real-time transparency into the model's autonomous tool invocations.

  5. Cross-Device Ecosystem Synchronization: Any meal or adjustment logged via ambient voice on Alexa instantly synchronizes with the companion mobile app diary and macro progress rings without requiring a manual refresh.

How I built it

Caliwo was architected from the ground up as an enterprise-grade, multi-service agentic pipeline:

  • Foundation Model Orchestration (Amazon Bedrock): Powered by Amazon Nova Pro (amazon.nova-pro-v1:0) via the AWS boto3 Bedrock Converse API. We implemented a 6-round autonomous agent orchestrator (backend/agent/orchestrator.py) capable of recursive tool selection, parameter extraction, and conversational reasoning.

  • Model Context Protocol (MCP Server): Engineered a self-hosted MCP server implementing the Model Context Protocol Spec 2025-11-25 over Streamable HTTP SSE transport (backend/mcp_server.py), exposing 9 standardized domain tools directly mapped to core health services.

  • Alexa+ Web Simulator (assistant/): Built with React 19, Vite, and Vanilla CSS design tokens. Features Web Speech API recognition, Web Audio API frequency analyzers for ambient waveform dynamics, and neural voice synthesis with audio caching.

  • Backend & Cloud Infrastructure: Built with FastAPI, containerized via Docker Compose, and deployed on an AWS EC2 Ubuntu instance. Utilizes Redis for pub/sub message broadcasting and sub-millisecond caching, backed by PostgreSQL with connection pooling and composite index optimization.

Challenges I ran into

  1. The FastMCP / ASGI Lifespan Conflict (Documented in FRICTION.md): Mounting the FastMCP Streamable HTTP server directly onto the root FastAPI pipeline caused a critical ASGI lifespan conflict—FastMCP’s internal lifespan handler was bypassed, blocking protocol session initialization. We engineered a custom lifespan multiplexer and reverse proxy routing layer, thoroughly documenting the issue and workaround in FRICTION.md as a contribution to the AWS and MCP developer communities.

  2. Cross-Session Conversational Context Resolution: Handling commands like "Add that" requires maintaining contextual state across separate turns without hallucinating previous recommendations. We implemented an in-memory recommendation cache backed by persistent conversation logs in PostgreSQL to cleanly track context IDs.

  3. Optimizing Database Roundtrip Latency: Connecting asyncpg to pooled PostgreSQL endpoints originally suffered from prepared statement cache conflicts in transaction pooling mode. We eliminated statement cache overhead (statement_cache_size=0), removed redundant pre-ping network roundtrips, and batched multi-step meal logs into single-transaction atomic commits.

Accomplishments that I'm proud of

  • True Agentic Workflow (Not a Simple Q&A Bot): Built a multi-turn autonomous agent loop that calls up to 9 MCP tools sequentially, reasons over combined outputs, and adapts responses dynamically based on real-time health data.

  • Comprehensive Test Suite (18/18 Automated Tests Passing): Authored an end-to-end integration test harness (backend/tests/test_e2e_agent.py) verifying multi-turn conversational tool calling, nutritional gap mathematics, and database commit integrity.

  • Seamless Cross-Device Data Plane: Achieved unified data consistency where voice commands spoken to the Alexa hardware simulator immediately reflect in the mobile application without lag or polling delays.

  • Bonus Points Contribution: Provided an actionable, production-tested friction log in FRICTION.md for Amazon's developer evaluation team.

What I learned

  • The Decoupling Power of MCP: Standardizing tool definitions using the Model Context Protocol completely decouples the LLM provider from backend microservices. Swapping or upgrading foundation models (such as testing Bedrock Nova Pro against Claude 3.5 Sonnet) required zero changes to our underlying business logic tools.

  • Voice-First Ergonomics: Designing for voice assistants requires a fundamentally different conversational philosophy than text chatbots: responses must be concise, warm, numbers must be contextualized rather than listed as raw data tables, and follow-up actions must be friction-free.

  • ASGI Lifespan Architecture: Deepened our expertise in Starlette and ASGI lifespan protocols when bridging independent protocol servers into a unified containerized API.

What's next for Caliwo (Ambient Nutrition Intelligence Hub)

  1. Official Alexa+ Skill Package Release: Package and publish our MCP agent directly to the Amazon Alexa Skill Store for deployment on physical Echo Show and Echo Dot hardware.

  2. Echo Show Computer Vision Integration: Leverage the camera on Echo Show devices to allow users to hold up a plate and say, "Alexa, what's on my plate?", pairing Bedrock Multimodal Vision with our nutrition gap tools.

  3. Smart Grocery Fulfillment: Connect the create_meal_plan tool to Amazon Fresh and Whole Foods APIs to automatically assemble shopping carts for recommended deficit-filling meals.

  4. Cross-Device Ambient Health Ecosystem: Integrate with wearable telemetry (Apple HealthKit / Health Connect) so Alexa dynamically adjusts dinner calorie budgets based on daytime step counts and active burn rates.

Built With

Share this project:

Updates

Submission history