Inspiration
Almost everyone who tracks calories and macros hits the same wall: the plan is easy, the cooking is hard. You open the fridge, you have chicken, rice, eggs and half a bag of spinach, and you need 600 kcal with at least 45 g of protein. Meal-planning apps hand you recipes you don't have ingredients for, and chatbots confidently invent nutrition numbers that look right and aren't.
Two things came together for me. First, the Alexa+ track asked for something more than a thin wrapper around an existing API: an agentic workflow that keeps state across sessions, can purchase, and shows rich cards. Second, my bachelor thesis is about detecting hallucinations in LLM output using knowledge graphs and symbolic rules. Meal planning is the perfect place to apply that idea, because a wrong number is a real-world mistake, not just a bad sentence.
So the question became: can an agent plan a meal from the food I actually own, and be provably right about the numbers?
What it does
You talk to Alexa+ (in my demo, a web simulation of Alexa+), and it:
- Plans a meal to your target. "Build me a 600 calorie dinner with at least 45 grams of protein." It composes a meal from your pantry, solves exact gram portions, and verifies the totals.
- Manages your inventory by voice. "I bought 500 grams of chicken breast, 1 kilogram of rice and 6 eggs." Free text becomes measured stock, and if a quantity is ambiguous it asks instead of guessing.
- Tracks your daily macros. Set daily targets, log a verified meal, and ask what you have left. Logging also decrements the matching pantry stock.
- Learns your preferences. Diet, allergies, and likes and dislikes feed back into later meal plans.
- Shops for what's missing, only with your approval. If your pantry can't hit the target, it builds a cart from a mock catalog and waits for an explicit yes before anything is ordered.
- Returns cards, not just speech. Each tool returns a spoken summary plus an MCP Apps card (meal, inventory, cart approval, nutrition status).
The key promise: a meal is labeled verified only if a deterministic check passed. Otherwise the system says so honestly and shows the closest attempt.
How I built it
The Alexa+ side is a self-hosted MCP server (Python, FastMCP, Streamable HTTP) exposing four goal-level tools: plan_meal, manage_inventory, shop_for_meal and nutrition_status. Behind plan_meal sits a multi-agent system built as a cyclic LangGraph state graph:
| Agent | Job |
|---|---|
| Supervisor | Decides the next step. A deterministic state machine defines which actions are legal right now, and an optional LLM router picks among them. |
| Inventory | Parses speech into grams, converts units, asks clarifying questions, reads receipts and barcodes, handles reservations and expiry. |
| Composer | Proposes ingredients and portion bounds, favoring items that expire soon and respecting preferences. It is not allowed to claim nutrition. |
| Verifier | Looks up nutrition (Neo4j, then USDA FoodData Central, then Open Food Facts), applies cooking-yield factors, solves portions, and checks diet and allergen rules. |
| Procurement | Computes the ingredient gap, builds a mock cart, and enforces the approval gate. |
Verification is math, not vibes. The LLM proposes which ingredients; it never outputs calories or macros. The portions $x_i$ (in grams) are found by a constrained optimization (SciPy/HiGHS) so that the totals land inside the requested ranges:
$$ \ell_i \le x_i \le u_i, \qquad \sum_i x_i \, \frac{k_i}{100} \in [K_{\min}, K_{\max}], \qquad \sum_i x_i \, \frac{p_i}{100} \ge P_{\min} $$
where $k_i$ and $p_i$ are USDA kcal and protein per 100 g, and $\ell_i, u_i$ are bounded by what's actually in stock. Results are then cross-checked for internal consistency with the Atwater relationship:
$$ \text{kcal} \approx 4P + 4C + 9F $$
If verification misses, the failure feeds back into the composer as concrete repair hints ("add about 12 g protein"), and the loop tries again within a fixed repair budget. If the pantry simply can't do it, procurement proposes a cart, and the meal is re-verified after the (mock) delivery.
Guardrails are enforced in code, not in prompts:
- A hard completion gate: nothing returns as
verifiedwithout a passing deterministic check. - Tool-call and wall-clock budgets, so the loop always terminates with an honest result.
- Shopping requires explicit approval, with idempotency and audit trails, and only a clearly fake catalog is connected.
- Per-session state in SQLite (optional Redis cache), so a clarification can be answered turns later with the same
session_id.
Testing without a device: I built a small web simulation of the Alexa+ conversation so the whole experience can be run and demoed from a browser, and I tested the tools directly with MCP Inspector.
Stack: Python, FastMCP (MCP, Streamable HTTP), LangGraph, SQLite, SciPy/HiGHS, USDA FoodData Central, Open Food Facts, Neo4j (optional knowledge graph), Pydantic, uv.
Challenges I ran into
- Keeping the LLM honest. The easiest design is to ask a model for a recipe plus its macros. It is also the one that quietly lies. Splitting the roles was the hardest design decision: the model proposes and routes, but facts come from trusted data, and a deterministic gate decides what can be called verified.
- Bounded autonomy. I wanted a truly agentic system, where the agent picks tools and plans, but a loop that can call tools forever is a liability in a voice product. The legal-actions state machine plus budgets gave me autonomy where it helps and determinism where it matters.
- Messy human quantities. "Two cups of rice", "a dozen eggs", "some spinach." Volume and counts need ingredient-specific densities, and some inputs are genuinely unanswerable. The inventory agent surfaces uncertainty as a question instead of guessing.
- Targets that underspecify the goal. A calories-only target is satisfied by almost any food, and a naive optimizer will happily reach it with olive oil. Defining what a "good" meal means when the user gives only part of the picture taught me that constraints need to encode intent, not just arithmetic.
- From ingredient lists to real meals. A list of gram amounts isn't a meal. Turning solved portions into something you can actually cook, without letting the model touch the numbers, is an ongoing design problem.
- Approval and state across turns. Resuming a conversation after a clarification or a cart approval means the graph has to remember where it stopped, without ever letting a stale "yes" trigger an order.
Accomplishments that I'm proud of
- A system where "verified" actually means something, from the data source to the final gate.
- A genuinely multi-agent workflow with state across sessions, purchasing with consent, and MCP Apps cards, rather than a single-turn Q&A bot.
- Bringing ideas from my thesis on hallucination detection (grounding model output in structured knowledge and symbolic rules) into a product people could use every day.
What I learned
- Agentic does not have to mean unconstrained. The best results came from letting the model choose among safe options.
- Voice changes everything: every tool needs a short spoken summary, clarifying questions must be rare and precise, and failures must be honest and brief.
- Data coverage is the quiet bottleneck. The quality of the verification is capped by the quality and coverage of the food data behind it.
- Treat tool outputs and recipe text as untrusted input, and keep side effects (like shopping) behind explicit, auditable approval.
What's next for the Alexa+ Meal & Macro Planner
- Alexa+ account linking and real per-user identity (today it runs as a single local demo user).
- Better handling of partial targets, so "just give me 600 calories" produces a balanced plate by default.
- Turning verified portions into short, voice-friendly recipes.
- Broader curated nutrition, yield and allergen data in the knowledge graph.
- A real retailer adapter, once the proper purchase authorization flow is in place.
- Interactive MCP Apps cards with a trusted approval signal.
Log in or sign up for Devpost to join the conversation.