Allergy Crosscheck

An Alexa+ add-on that tells you who at the table can eat a dish or a scanned food, and why. Its results never say "safe": when the data is missing, they say "I can't confirm".

Comment

Who at the table can eat tonight's dinner, and why? Allergy Crosscheck answers that by voice and on the Echo Show screen, for the whole household and its guests, from a recipe or a scanned product. When the data can't back an answer, it says "I can't confirm" instead of guessing.

What is real and what is simulated: the MCP server, the allergen engine, the Echo Show screen and the phone scan are the shipped code; the Alexa+ host in the video is a simulator driven by a local model, because Amazon's Alexa+ add-on tools are a partner-only preview that hackathon participants can't access (hackathon FAQ; details under "Simulated Alexa+" below).

For the judges, in their four criteria

  • Tech implementation: self-hosted MCP server, spec 2025-11-25 over Streamable HTTP, 9 tools, MCP Apps screen with tap-to-choose; 207 automated tests, 100 in the engine.
  • Design: one ask gives a verdict per person on the Echo Show screen and in speech, with the risky ingredients and swaps; guests expire; the screen is authored at the guide's 768×480 base canvas.
  • Potential impact: 1 in 15 US adults has a food allergy. A real cocoa mix lists milk only as "may contain" while its ingredients hold nonfat dry milk, whey and caseinate; this catches it. 72.3% of the 4.8 million Open Food Facts records, and 53.2% of the products scanned in its app, carry no allergen data and no ingredient list, so the only honest answer is "can't confirm", and that is the answer, never "allergen-free". Among app-scanned products with English ingredient text, it gives a decisive answer for 56 to 71% of records; across all scanned products sold in the US, 37.2 to 46.6%.
  • Quality of the idea: three of Amazon's own "creative" examples built (agentic workflow in one turn, state across sessions, MCP Apps), and a cross-check that catches a real cocoa mix whose tags say milk is only "may contain".

Inspiration

I have seen people close to me go through allergic reactions, and I have had one myself without ever finding out what caused it. That is where this started.

Then two real records from Open Food Facts, the open database behind many allergy apps. A chocolate hazelnut spread has an empty allergen list and no ingredient list, so a checker that trusts the list would call it allergen-free. A hot cocoa mix lists soy as its only allergen and milk only as a "may contain", while its ingredients include nonfat dry milk, whey and caseinate. A voice assistant that answers confidently from data like that is dangerous. One that knows when it doesn't know is useful.

This is common. I ran the engine over all 4,784,317 Open Food Facts records (snapshot of 7 October 2026). 72.3% of them list no allergens, no "may contain", and have no ingredient list. So do 53.2% of the products scanned in the Open Food Facts app. A checker that trusts the list would say "no allergens" for each one; this one says "can't confirm". Details: packages/allergen-guard/bench/README.md in the repository. Data: Open Food Facts (ODbL).

The stakes are high. In the US, about 1 in 15 adults (6.7%) and 1 in 19 children (5.3%) have a food allergy (CDC, National Health Interview Survey, 2024 data). In national surveys, 42% of US children with a food allergy had been to an emergency room for it at least once (Gupta 2018), and 51% of adults with one had had a severe reaction (Gupta 2019). Two situations stand out: in a Dutch study of unexpected reactions, 44% came from pre-packaged products and 23% from meals eaten away from home, at friends' or relatives' homes, at school or in restaurants (preliminary results, Michelsen-Huisman 2015). "May contain" labels don't settle it either: 46% of the food-allergy shoppers surveyed were unsure or wrongly believed those labels are required by law (Marchisotto 2017). At home, the real question is whether a dish is OK for everyone at the table tonight.

Alexa+ already remembers a family's allergies. Amazon says it understands "your family's dietary preferences, allergies, and weekly traditions" (Amazon). But remembering an allergy isn't the same as checking food for it.

What it does

Allergy Crosscheck tells you who at the table can eat a dish or a product, and why. It checks recipes and scanned products against everyone at your table, guests included, and cross-checks a product's allergen list against its ingredient text. It treats missing data as unsafe: it says "I can't confirm" instead of "allergen-free". Among the 264,667 app-scanned Open Food Facts products that carry English ingredient text, that gives a decisive answer (avoid, or no listed allergens) for 70.1% of records with a milk allergy, 56.1% with peanut and tree nuts, and 70.8% with milk and celery (measured on 14 October 2026, details in the engine's bench README).

Say Maya can't have peanuts or tree nuts, and Leo from next door, who can't have dairy or celery, is coming on Saturday. "Check tonight's lasagne" gives each of them a verdict on the Echo Show, names the risky ingredients, and suggests swaps checked against both of them, so a fix for Leo never trips Maya.

  • Household and guests: who is allergic to what, with an end date for visitors. Guests stop being checked after their last day, unless asked for by name.
  • Recipes: a dish name, looked up on TheMealDB, or ingredients read out. You get a verdict per person, the risky lines, and household-aware swaps that go onto a shopping list from the same ask.
  • Packaged food: scan the barcode on your phone and ask "Can Maya have what I just scanned?", or say the name or the barcode. If several products match, it asks which one. It reads the Open Food Facts allergen tags, traces and ingredient text together, which is how it finds the cocoa mix's milk.
  • Fail-closed: there are three answers: "Avoid", "Can't confirm" and "None listed, check the label". Never "safe". Missing, ambiguous or non-English data gives "Can't confirm". So does a recipe word it doesn't recognise ("I don't know what's in garam masala") and a failed lookup. An unnamed "vegetable oil" or "flavouring" stays "Can't confirm" too, because either can carry an allergen the label never names; it is shown as a line to check rather than waved through.

How I built it

  • More than an API wrapper. Amazon names a "basic MCP wrapper around an existing API" as the obvious Alexa+ idea (rules). Of the creative examples in the judging criteria, this builds three (an agentic workflow in one turn, state across sessions, MCP Apps) and part of one more (cards):
    • State across sessions: the household and dated guests are saved between conversations, and after their last day a guest stays listed, marked as ended, and is no longer checked unless asked for by name (src/server/store.ts, until; save_person in src/server/tools.ts). The "what I just scanned" scan is kept for 30 minutes.
    • Agentic workflow, in one turn: one ask, "Check the lasagne for everyone, then add the swaps to my shopping list." (video 0:53), makes the model run check_recipe, read the swaps from its result and call add_to_shopping_list with exactly those three items, with no second prompt. Inside check_recipe the server, not the model, looks the dish up on TheMealDB, checks every ingredient for every person and proposes swaps checked against everyone; that part is deterministic code (src/server/tools.ts, src/server/sources.ts). check_product fetches packaged food from Open Food Facts the same way.
    • MCP Apps: the Echo Show screen, with tap-to-choose that calls back into the server (public/widget.html).
    • Cards, partly: the screen is made of card panels: "Who's eating", "Watch out for" (or "What I found" for a product) and "Swaps", plus a choice screen and a shopping-list screen. These are HTML sections I drew inside the MCP Apps view, not Alexa+'s own card or carousel components, and there are no images or media.
    • Not built: purchasing, and Agent Skills.
    • A cross-check, not a lookup: a product's allergen tags, traces and ingredient text are read together, which is how the cocoa mix's "may contain milk" becomes a firm "contains milk" with the ingredients named.
  • Self-hosted MCP server in TypeScript with the MCP TypeScript SDK 1.32. It uses MCP spec 2025-11-25 over Streamable HTTP, uses a stateless Streamable HTTP transport (the household is stored server-side), and has 9 tools. Every verdict result (check_recipe, check_product) leads with a VERDICT: line (UNSAFE, CANNOT CONFIRM or NO LISTED ALLERGENS: the keys behind "Avoid", "Can't confirm" and "None listed"), and every result the server builds carries the exact sentence to say, plus structuredContent for the screen.
  • MCP Apps screen for the Echo Show (text/html;profile=mcp-app), following the Alexa+ MCP design guide's base canvas, zoom, touch-target and contrast figures, and using the hostContext fields theme, safeAreaInsets, maxWidth and maxHeight:
    • authored at the guide's 768×480 base canvas with its type, spacing and corner-radius tokens; the widget applies one root scale from hostContext maxWidth/maxHeight, 1.667 for the Echo Show 8's 1280×800 (guide figures), and each demo verdict fits both with no scrolling (measured in headless Chrome, not on a real Echo Show);
    • text is 14px or larger at the base canvas (my floor; the guide's type table goes lower), touch targets at least 48px tall at the base canvas (50px as rendered; guide minimum: 48×48px), text contrast at least 4.5:1 (guide: 4.5:1 for text 17px and under, 3:1 for bold or 18px and larger);
    • it uses the host's theme and safe-area insets.

Tapping a choice calls back into the server with tools/call. _meta.ui.resourceUri is on every tool definition and every result the server builds.

  • Voice design: every spoken verdict names who, what and up to three of the lines it came from, because on a screenless Echo the sentence is all there is; the screen carries the full list. A verdict for two people runs about 8 to 14 seconds in the video; a shorter line would drop the ingredient names, which is the part a listener acts on.
  • Open-source engine, allergen-guard, MIT, with zero dependencies. It covers the EU 14 and US Big 9 allergens, with wheat separate from gluten, and has an ingredient dictionary with exclusions (peanut butter isn't dairy; oyster mushrooms aren't shellfish). Swaps declare their own allergens and are never a plant relative of what they replace. Coconut swaps (coconut cream for crème fraîche) are not treated as tree nuts: coconut is not among the EU's 14 allergens, and the FDA dropped it from its tree-nut list in its January 2025 allergen labeling guidance.
  • Phone scan page: the camera reads the barcode with the browser's BarcodeDetector, or a WebAssembly decoder that the server serves itself, for browsers without it, such as Safari. The server keeps the scan for 30 minutes as "what I just scanned".
  • Simulated Alexa+, because Amazon's Alexa+ add-on tools (MCP Toolkit, CLI, web simulator) are a preview for select partners, with no way for hackathon participants to get access (hackathon FAQ); the MCP Toolkit overview also limits it to the United States. An LLM drives the server through a real MCP client, and the browser handles voice and hosts the MCP Apps screen. In the video, Alexa's lines are Amazon's own Alexa voice, rendered in the developer console from the exact text the run returned; the narrator and the user are synthetic voices (Qwen3-TTS). The simulator adds two guards, as a test harness rather than part of the add-on:
    • a verdict lock: after a check or a change, the spoken answer is the tools' own sentences, verdicts first;
    • a no-tool guard: a keyword filter that blocks common verdict and action phrases when no tool has returned a result.

On a real Alexa+, add-ons can't script speech. What ships is the server's side: every verdict result starts with a VERDICT: line, and every result the server builds carries the exact sentence to say. A verbatim hint for safety answers is one of the two critical items in the product feedback's feature requests.

  • 207 automated tests (100 of them in the engine package), including:
    • 85 ingredient–allergen pairs that must never pass (peanut butter, pesto, ghee, amaretto…);
    • look-alikes that must not be flagged;
    • MCP end to end over HTTP;
    • a fake model that tries to say "totally safe" and is overridden.
  • Built with Claude Code as the coding agent.

What was and wasn't tested

  • Tested:
    • the 207 automated tests;
    • the full demo conversation, end to end in the simulator, on a local Qwen3.5 9B model with live data;
    • the scan page in headless Chrome, with a camera feed of a real barcode;
    • every demo screen, measured in headless Chrome at 768×480 (the guide's base canvas, dark and light) and at 1280×800 with the 1.667 root scale (dark and light), and in the simulator's Echo Show 8 frame.
  • Not tested:
    • a real Alexa+: the tools are a partner-only preview that participants can't access (hackathon FAQ), so there was no deployment to test;
    • Amazon's Local Inspector. Even with a developer account, it installs from a private registry that needs an AWS account with access to an Amazon role (friction log 4).

Challenges I ran into

  • Open data that contradicts itself. The answer was never to trust allergen tags alone, and to scan the ingredient text too.
  • Words the engine didn't know. Running it over Open Food Facts found six: "gluten", "milkfat", "butterfat", "crustaceans", "cereals containing gluten" and "E322". They are fixed now, each with a test.
  • Category tags and unreadable ingredients. Open Food Facts also tags each label ingredient with its parent categories (rice flour is also "flour", peanut also "nut"), and read as words those added allergens the label never named. The engine now reads the label's own tags and skips a short list of generic categories: 57,571 records lose a false "contains". And when part of an ingredient list is not in English, it now says "can't confirm" instead of "none listed": 55,933 records (both figures from the 8 October run, on an earlier engine). The records whose ingredient text names an allergen their tags omit number about 8,300 by the bench's strict count on English text (about 2,070 of them app-scanned). In the last check of that count, 100 of them were split between two independent AI agents, 50 each, with a re-check: 99 of the 100 really omit the allergen, so the count is not claimed as exact. The opposite check ran on 13 October: when the engine says "no listed allergens", does the ingredient text name one of the person's allergens after all? 100 such answers, drawn at random from scanned products with English ingredient text for three household profiles and checked the same way, gave none; 0 in 100 is consistent, at 95% confidence, with a miss rate of up to 3.7%, and in 6 of the 100 the "ingredient text" was a nutrition panel or marketing text, not a list. The engine now refuses such text: a nutrition panel or a run of words that names no food gives "can't confirm", which moved 1,407 of the 264,667 records out of "no listed allergens" (bench README).
  • Plant-based names that contain dairy words ("coconut yoghurt"). Swaps declare their own allergens instead of being re-parsed.
  • Voice answers have to be short without losing the reason. Every verdict says who, what, and where it came from.
  • A real model behaves differently from a scripted one. It said "Got it, I'll add Leo" without saving him, and answered a pesto question from memory; that led to the no-tool guard. It also made up shopping-list swaps until the turn history kept the tool results.

Accomplishments that I'm proud of

  • The server's results never call a food "safe"; the engine's tests check that, and a simulator test checks that a fake model's "totally safe" is overridden. On a real Alexa+ the spoken words are the model's.
  • One spoken question checks everyone at the table, including a guest who drops off after the visit.
  • In a real cocoa mix whose tags carry milk only as "may contain", it shows the milk is in the ingredients.
  • It follows MCP spec 2025-11-25 over Streamable HTTP, with an MCP Apps screen authored at the Alexa+ 768×480 base canvas and checked in an Echo Show 8 simulator frame (untested on a real Echo Show).
  • The language model runs locally (Qwen3.5 9B, 4-bit, MLX), with no cloud LLM API; product and recipe data come live from Open Food Facts and TheMealDB.

What I learned

  • An allergen list is a claim, not a fact. The ingredient text has to be read too.
  • A model with tools still skips them. The guard has to be in code, not only in the prompt.
  • Swaps need the same care as dishes. An early swap table offered fennel for celery, which is a plant relative and a known cross-reaction. Swaps are now tested against that.
  • On a voice device the screen has to read at a glance. Grouping the cocoa mix's 11 flagged ingredients and its "may contain" warning into 4 rows mattered more than any colour.

What's next for Allergy Crosscheck

  • Deploying through the Alexa+ MCP Toolkit once it's available to my account.
  • Scanning with the Echo Show's own camera, if add-ons get camera input.
  • Ingredient text in more languages: Open Food Facts carries it, and the engine needs dictionaries.
  • An allergist review of the dictionary and the swaps.

Built With

  • alexa+
  • barcode-detector
  • claude-code
  • mcp-apps
  • mlx
  • model-context-protocol
  • node.js
  • open-food-facts
  • qwen
  • streamable-http
  • themealdb
  • typescript
  • web-speech-api
Share this project:

Updates

Submission history