FoodHub

Inspiration

We started with a simple question: why is ordering from an unfamiliar menu still difficult even when translation tools already exist?

Our initial assumption was that diners mainly lacked context — not just the translated name of a dish, but information such as ingredients, taste, texture, cooking style, and dietary suitability.

Instead of building immediately, we spoke to diners from different cultural backgrounds about how they navigate unfamiliar menus and what information actually affects their decisions. We also examined existing tools such as translation apps and general-purpose AI.

One finding changed our direction: general-purpose AI could already explain unfamiliar dishes surprisingly well. The remaining problem was not simply access to information — it was turning that information into a fast, trustworthy decision.

That shifted FoodHub from a “better menu translator” into a decision-support experience for unfamiliar food.

What it does

FoodHub lets diners upload a photo of a restaurant menu and transforms it into structured, easy-to-scan dish cards.

Instead of searching for unfamiliar dishes one by one, users can quickly understand:

• what the dish actually is
• its main ingredients
• taste and texture
• relevant dietary concerns
• whether it fits their personal preferences

Users can also create a preference profile containing allergies, dietary restrictions, religious restrictions, preferred flavours, textures, proteins, spice tolerance, and disliked ingredients.

FoodHub treats dietary constraints differently from taste preferences. Safety-related restrictions are considered first, while softer preferences are used to help users narrow down and compare the remaining options.

The goal is not for AI to decide what someone should eat, but to give them enough context to make that decision confidently themselves.

How we built it

FoodHub uses a React and TypeScript frontend with a central FastAPI backend.

The frontend handles menu upload, user preferences, and structured dish-card presentation. The backend manages authentication, user profiles, recommendation logic, external food knowledge, and menu-processing orchestration.

For menu understanding, we use OpenAI vision and language models to convert menu images into structured data such as dish names, translations, prices, descriptions, and explicitly listed ingredients.

A key technical decision was to keep different types of evidence separate:

• Explicit evidence — information directly written on the restaurant menu
• Reference evidence — information from external food or recipe sources
• Inferred evidence — culinary information inferred by AI

We do not treat inferred ingredients as confirmed restaurant facts.

Unknown dishes can be enriched through external references or AI fallback, and multiple unknown dishes are processed together in batches to reduce unnecessary model calls.

Dietary and allergen checks are handled through backend rules rather than asking an LLM to make unrestricted safety decisions.

Challenges we ran into

One of the biggest challenges was realising that AI capability itself was not our main differentiator.

General-purpose AI could already translate menus, explain dishes, and even provide personalised suggestions. That forced us to rethink where FoodHub could actually add value.

Another challenge was uncertainty. A model may know that a traditional version of a dish often contains a certain ingredient, but that does not mean a particular restaurant uses it. We had to design the system so that restaurant evidence, external reference knowledge, and AI inference were never presented as if they had the same level of certainty.

We also had to coordinate several moving parts — frontend interaction, menu extraction, user preferences, external recipe matching, recommendation rules, caching, and AI fallback — while keeping the architecture simple enough for a hackathon prototype.

Accomplishments that we're proud of

We are especially proud that the product changed as a result of our research rather than staying fixed around our original idea.

We moved away from building another AI menu translator and instead focused on structured comparison and decision support.

On the technical side, we built the core components for:

• structured menu extraction from images
• menu translation
• reusable AI menu-understanding services
• user preference profiles
• unknown-dish fallback
• batch LLM processing
• separation of explicit, reference, and inferred evidence
• rule-based dietary and recommendation logic

We are also proud of designing the system to communicate uncertainty instead of hiding it. When the system cannot confidently know something about a restaurant's preparation, that uncertainty should remain visible.

What we learned

The biggest lesson was that translation and understanding are not the same thing — but understanding and decision-making are not the same thing either.

At first, we thought the main opportunity was to provide more information about unfamiliar dishes.

Our research showed that information is already widely available through search and AI. The more interesting problem is how that information is organised, compared, personalised, and presented at the moment someone needs to make a choice.

We also learned that AI should not be treated as a single source of truth. For food, allergies, and dietary restrictions, the source and certainty of information matter just as much as the information itself.

Most importantly, we learned that good product decisions came from challenging our own assumptions rather than trying to validate them.

What's next for FoodHub

Our next step is to continue testing FoodHub with real diners and measure whether it actually improves decision-making compared with their current behaviour.

We want to evaluate factors such as decision time, confidence, understanding, and the number of external searches needed before choosing a dish.

We also want to improve dish matching and reference coverage, expand support for more cuisines and menu formats, and make uncertainty even clearer for dietary and allergy-related information.

Longer term, FoodHub could learn from repeated user choices and evolve from a one-time preference profile into a more adaptive food preference model.

The goal remains simple:

help diners know what they are actually ordering — and make a choice they feel confident about.

Built With

• alembic
• api
• css
• fastapi
• mysql
• openai
• python
• react
• tailwind
• themealdb
• typescript
• vite