-
-
Scan
-
Preferences
-
Smart Menu
-
Dish details
Inspiration
You are in Rome. The menu is in Italian. You are keto.
Google Translate gives you words. It tells you that trofie al pesto is "trofie with pesto." It does not tell you that it is 76 grams of carbohydrate, or that pesto contains pine nuts even though the menu never says so.
Anyone eating to a rule is doing two translations at once: the language, and then the ingredients. The second one is the one that actually matters, and it is the one no tool does.
How we built it
A vision model reading a menu does not need OCR as a separate step. Handing it pre-extracted text would actually lose information: layout is what tells you which line is a section header and which is a dish.
The model does perception; deterministic code does judgment. That is why the app can explain itself.
Challenges we ran into
The model we planned on was retired mid-build. gemini-2.5-flash started returning 404 hours in. Then Gemini 3.x had replaced thinkingBudget with thinkingLevel, and the error was a bare "Request contains an invalid argument" with no field name. Our first real photo would have killed the demo. Vercel rejects request bodies over 4.5 MB before your code runs. We would have found that on stage, with a judge's menu. Our keto scoring was confidently wrong. It keyed on category === "pasta", so a risotto at 78 g of carbs showed as "Recommended" to a keto user, the exact opposite of the product's promise. The processing screen lied. It navigated on a timer after about 4 seconds, while the real call takes 10–20, so users saw our sample menu as though it were their own photo.
What we learned
Testing the API is not testing the product. Nearly every serious bug was invisible from the endpoint and obvious the second we drove the real flow in a browser at phone width.
Green CI does not mean safe to merge. One PR would have deleted 564 lines of model integration and still passed CI, because reverted code still compiles.
