Inspiration
We are three Koreans currently living and interning in the United States. Exploring local restaurants in a country where food cultures from around the world come together has been one of the most exciting parts of our experience abroad. However, every time we opened an unfamiliar menu, we encountered the same problem.
Google Maps and Yelp could help us decide where to eat, but once we arrived at a restaurant, they offered little help in deciding what to order. Translating a menu told us the literal meaning of a dish’s name, but it rarely explained what ingredients it contained, how it tasted, what its texture was like, how spicy it might be, or which parts of the description were uncertain.
We could ask a general-purpose AI tool, but doing so required entering menu items one by one and asking several follow-up questions. Even after that, it was not always clear whether the answer described the specific restaurant’s dish or only the typical version of that food. As a result, we often chose the safest or most familiar option instead of trying something new.
That experience revealed a gap: many services help people decide where to eat, but few help them understand the menu and decide what to eat after they arrive.
To solve a problem we had experienced ourselves, the three of us—none of whom had professional software development experience—formed a team with GPT-5.6 and Codex. Our real-world experience defined the problem, GPT-5.6 helped us turn it into a product and system design, and Codex helped us transform that design into working software.
What it does
Foodseyo helps travelers understand unfamiliar restaurant menus before they order.
Users can begin with a menu photo, restaurant photo, screenshot, or link. Foodseyo analyzes the available clues, finds likely restaurant locations, and allows the user to confirm the correct branch instead of automatically trusting the first or nearest search result.
It then transforms menu information into a structured interface that can explain:
- What each dish is
- Its basic taste and flavor
- Its expected texture
- Its spice and richness levels
- Important ingredients
- Cooking methods
- Restaurant-specific information
- General knowledge about the dish
- Missing or uncertain information
Foodseyo is not designed to simply translate menu text. Its goal is to give users enough cultural and culinary context to make a confident ordering decision without having to conduct a long AI conversation.
Restaurant identification improves the result, but it does not block the experience. If the exact restaurant or branch cannot be safely confirmed, Foodseyo continues in a menu-only mode and provides general dish explanations without attaching unverified prices, ingredients, or preparation details from another restaurant.
How we built it
We built Foodseyo using GPT-5.6, Codex, the OpenAI Responses API, Google Places, Next.js, and TypeScript.
GPT-5.6's multimodal capabilities are used to understand menu images, restaurant clues, unfamiliar dish names, and cultural food context. Google Places acts as an external evidence layer that provides possible real-world restaurant locations.
A restaurant search result is never automatically treated as the correct answer simply because it is ranked first or located nearby. Candidates remain candidates until the user confirms the correct location or sufficient supporting evidence becomes available.
The menu analysis follows a structured workflow:
Input acquisition
→ restaurant candidate resolution
→ GPT-5.6 menu analysis
→ schema validation
→ vocabulary normalization
→ evidence and uncertainty separation
→ structured menu explanation
Instead of displaying or storing an unrestricted AI response, Foodseyo converts the analysis into consistent categories such as taste, flavor, texture, spice, richness, ingredients, and cooking methods.
We also separated three important domain concepts:
- Restaurant: one real physical restaurant location
- MenuItem: an item sold in a particular restaurant and menu context
- Dish: general food knowledge that may be shared across restaurants
This prevents general culinary knowledge from being presented as a verified fact about a specific restaurant. For example, an ingredient that is commonly used in a traditional dish is not automatically described as an ingredient used by the restaurant currently being viewed.
Information explicitly shown on the menu has the highest priority. Reasonable inference is treated separately, and general dish knowledge is used only as supporting context. When there is not enough evidence, Foodseyo preserves the value as unknown rather than converting it into “none,” “safe,” “false,” or zero.
We also defined an exact analysis identity using the input, restaurant context, model, prompt, schema, normalization rules, and policy versions. This means a previous result can be safely reused only when the conditions that produced its meaning are also the same.
GPT-5.6 and Codex were essential not only inside the product, but throughout the development process.
GPT-5.6 helped us:
- Convert a personal frustration into product requirements
- Separate important domain concepts
- Identify unsafe assumptions and data boundaries
- Design the user flow and development sequence
- Interpret review findings and technical tradeoffs
Codex helped us:
- Implement features and tests
- Work across isolated branches and worktrees
- Review pull requests at exact commit versions
- Resolve integration conflicts
- Run validators and continuous integration checks
- Turn our system design into a working application
Challenges we ran into
Turning flexible AI explanations into reliable product data
Our original idea sounded simple: upload a menu, ask GPT-5.6 to explain the dishes, and display the answer in a better interface.
However, the same food could be described as “spicy and savory,” “rich with moderate heat,” or “slightly hot with a deep umami flavor.” These may communicate similar ideas to a person, but an application cannot reliably compare, search, store, or reuse them as the same information.
We solved this by treating GPT-5.6's output as an intermediate reasoning result rather than the final product data. Taste, flavor, texture, spice, richness, ingredients, evidence, and uncertainty are validated and normalized into a consistent structure before being presented to the user.
Separating general dish knowledge from restaurant-specific facts
One of the most dangerous possible errors was telling a user that a restaurant's dish contained an ingredient simply because that ingredient is commonly used in traditional versions of the food.
We separated general Dish knowledge from restaurant-specific MenuItem facts. General knowledge can explain what a dish is usually like, while actual ingredients, prices, availability, and customization options require evidence from the specific restaurant and branch.
Handling restaurant ambiguity without blocking the user
Menus often show only a logo, contain no restaurant name, or belong to chains with several nearby locations.
Automatically selecting the first Google Places result could connect the menu to the wrong restaurant. Requiring exact identification, however, would make the entire experience fail even when GPT-5.6 could still explain the food.
Our solution was menu-only continuation. If the restaurant cannot be safely confirmed, Foodseyo continues to provide general dish explanations while withholding unsupported restaurant-specific details.
Collaborating without professional development experience
All three team members were new to professional software development. We initially thought dividing the features among ourselves would be enough, but differences in meaning were more dangerous than ordinary code conflicts.
For example, one developer could interpret “confirmed” as “selected by the user,” while another could interpret it as “verified by external evidence.” The code might still compile, but uncertain information could be displayed as fact.
We therefore defined shared contracts for restaurant candidates, confirmed restaurants, menu items, dishes, evidence, unknown values, cancellation, and external provider boundaries.
We also used separate branches, worktrees, packages, pull requests, exact-commit reviews, validators, continuous integration, and post-merge verification to keep independently developed features consistent.
Accomplishments that we're proud of
We are proud that Foodseyo became more than a menu translator or restaurant finder.
We created an end-to-end experience that can begin with an unstructured menu image, identify possible real-world restaurant locations, let the user confirm the correct branch, and transform menu information into practical explanations designed for an ordering decision.
We are particularly proud of how Foodseyo handles uncertainty. It does not automatically select the first restaurant candidate, convert missing information into a confident answer, or present general food knowledge as a verified restaurant fact.
Even when a restaurant cannot be confirmed, Foodseyo continues helping the user at the safest available level.
Most importantly, three people without professional development experience were able to design and build an AI-native product during one intensive week. GPT-5.6 and Codex did not simply help us write code faster. They allowed us to reason about product requirements, system boundaries, testing, collaboration, and user trust at a depth that would otherwise have been difficult for our team to reach.
What we learned
We learned that the real problem was not simply translation. The real challenge was transforming incomplete and uncertain menu information into something a person could trust enough to make a decision.
We learned that a fluent AI response is not automatically reliable product data. Model output needs structure, validation, normalization, provenance, versioning, and clearly defined boundaries before it can become part of a dependable application.
As first-time developers, we also learned that effective collaboration depends on shared meanings more than shared files. Contracts, ownership boundaries, exact-commit reviews, validators, and post-merge checks helped us prevent semantic conflicts that normal compilation could not detect.
Finally, we learned that GPT-5.6 and Codex are most powerful when they are treated not as automatic answer machines, but as collaborators operating within clear human-defined goals and constraints.
Foodseyo began with a simple question we repeatedly asked ourselves in unfamiliar restaurants:
“What should I order?”
Through this project, we learned that answering that question well requires more than translation. It requires context, evidence, uncertainty, thoughtful design, and trust.
What's next for Foodseyo
Our next goal is to expand Foodseyo from a hackathon MVP into a service that travelers can use in real restaurants.
We plan to:
Complete the production persistence and publication pipeline Expand the number and variety of supported menu sources Improve restaurant and branch identification Accumulate reusable, evidence-backed dish and menu knowledge Add dietary preference, allergy, and spice-tolerance settings Introduce a Food Passport that records the dishes users have tried, the cuisines they have explored, and how their tastes evolve over time Provide personalized menu comparisons and recommendations Recommend complete group meal combinations and ordering courses based on the number of diners, shared preferences, dietary needs, and spice tolerance Add multilingual explanations for travelers from different countries
Ultimately, we want Foodseyo to become a trusted decision-making layer between discovering a restaurant and ordering a meal—helping individuals and groups explore unfamiliar food cultures with less uncertainty, make better choices together, and build their own Food Passport along the way.
Built With
- codex
- github
- github-actions
- google-places
- gpt-5.6
- neon
- next.js
- node.js
- openai
- openai-responses-api
- postgresql
- react
- typescript
- vercel
- zod
Log in or sign up for Devpost to join the conversation.