xilo-Food Inspector

Inspiration

Packaged food labels contain important information, but understanding ingredients, nutritional values, and potential allergens can be confusing. Many people spend time searching online to understand what they are eating, especially when they have specific dietary preferences or restrictions.

We wanted to make this process simpler by building Food Inspector, an AI-powered application that helps people understand food labels and make more informed choices.

What it does

Food Inspector transforms a photo of a packaged food label into easy-to-understand insights.

• AI Food Scanner:Upload a food-label image for analysis.
• Ingredient Analysis: Understand ingredients in simple language.
• Nutrition Insights: Review available nutritional information.
• Personalized Checks: Identify potential conflicts with user-provided allergies, dietary restrictions, and preferences.
• Live Web Verification: Use TinyFish to search for relevant product information and retrieve webpage content for cross-checking.
• Shareable Reports:Share or download food analysis reports.
• Voice Summary:Listen to a summary of the analysis.

Food Inspector is designed to provide informational guidance, not medical diagnosis or a guarantee that a product is safe.

How we built it

We built Food Inspector as a full-stack web application using Next.js, React, TypeScript, and Tailwind CSS.

We integrated Gemini 2.5 Flash through OpenRouter to analyze uploaded food-label images and generate structured food analysis. The results include available ingredients, nutritional information, potential concerns, and personalized observations.

We integrated TinyFish Search and Fetch APIs as a live web research layer. After identifying a product, the application can search for relevant online information and retrieve content from suitable webpages to support cross-checking against the uploaded label.

We also developed a results dashboard, personalized user profiles, report-sharing functionality, and browser-based voice summaries. API credentials are stored in server-side environment variables rather than exposed in the frontend.

Challenges we ran into

One major challenge was extracting useful information from food-label images, which can contain small text, unfamiliar ingredient names, and complex nutrition tables.

We also had to handle API failures and incomplete AI responses without breaking the application. Another challenge was matching online product information to the exact product shown in an image, since different variants may have different ingredients or nutritional values.

We addressed these challenges by separating image analysis from web verification, implementing error handling, and ensuring that unavailable verification does not automatically prevent the main analysis from working.

Accomplishments that we're proud of

• Built a working AI-powered food-label analysis workflow.
• Integrated Gemini 2.5 Flash through OpenRouter for image-based analysis.
• Incorporated TinyFish for live web search and webpage retrieval.
• Created a personalized analysis experience based on user-provided dietary needs.
• Developed shareable food-analysis reports.
• Designed the application to be responsive, understandable, and extensible.

We are proud to bring food-label analysis and web-based product research together in one simple experience.

What we learned

We learned how to integrate AI vision capabilities into a full-stack application, structure AI-generated responses, handle API errors, and combine live web research with image-based analysis.

We also learned that AI-powered applications need to communicate uncertainty clearly. A product found online may not be the exact product or formulation in the uploaded image, so verification should support the analysis rather than be treated as unquestionable proof.

What's next for Food Inspector

We plan to expand Food Inspector with:

• Barcode Scanning: Identify products through their barcodes.
• Multilingual Ingredient Analysis: Support food labels in Hindi, Marathi, and other languages.
• Long-Term Dietary Insights: Help users understand patterns across their saved food analyses.
• Improved Product Verification: Better match products and compare ingredient and nutrition information across sources.
• Enhanced Reports: Make it easier to share food-analysis reports with family members and healthcare professionals.

Our vision is to make food information easier to understand and help people make more informed food choices, one scan at a time.

Built With

• next.js
• openrouter
• react
• tailwind
• tinyfish
• typescript