Cafeterian Lens

Home Page
Analyzing the tray
Trend

Inspiration

Cafeteria Lens was inspired by the need to address food waste in institutional settings. Every day, school and corporate cafeterias discard tons of nutritious food, but tracking the difference between what is consumed versus what is thrown away has historically required tedious manual auditing,posing a difficulty in customizing the menus according to the needs.

What it does

Cafeteria Lens is an AI-powered environmental and nutritional intelligence system designed to minimize food waste and optimize meal portioning. By analyzing photos of returned lunch trays, the system performs the following: 1)Identifies and classifies food items: It detects main dishes, sides, vegetables, fruits, beverages, and condiments. 2)Quantifies discarded mass: It estimates the initial fresh weight and the remaining mass of each item using structured JSON vision outputs. 3)Generates actionable insights: It provides constructive commentary to help school boards and kitchen managers re-engineer menus, adjust portion sizes, and reduce carbon footprints.

How we built it

Cafeteria Lens is built as a robust, full-stack application with a modular architecture.
1)Frontend: React 18, Vite, and Tailwind CSS are used for modern, adaptive layouts that display charts, filters, and logs. 2)Backend: The system runs on an Express Node.js server using TypeScript with automatic native compilation. 3)AI Analytics: It leverages the @google/genai TypeScript SDK to deploy structured multimodal schema parsers, ensuring consistent, machine-readable JSON results. 4)Database: A local, persistent SQLite engine records historical audit logs, utilizing tray_records and food_items tables for traceability. 5)Data Export: Internal client-side CSV parsing algorithms are used for offline record-keeping.

Challenges we ran into

Achieving Consistent Vision Accuracy: Handling varied lighting conditions and food overlapping on lunch trays made it difficult to accurately segment and classify items during the initial development phase.
Enforcing Strict Structured Outputs: Ensuring the AI consistently returned data conforming to the specific JSON schema required extensive tuning of the Gemini model to prevent parsing errors. Quantifying Discarded Mass: Calibrating the model to accurately estimate the initial weight and remaining percentage of diverse food types into grams presented a significant technical hurdle.

Accomplishments that we're proud of

Building an End-to-End Audit Trail: We successfully implemented a full-stack link from the initial image capture to persistent SQLite storage, ensuring complete traceability for every scanned tray.
Creating an Actionable Data Loop: We integrated AI analytics with a database to turn raw visual data into constructive nutritional insights, effectively replacing manual, tedious auditing processes. Robust Modular Architecture: We successfully leveraged the @google/genai SDK alongside a React frontend to build a modern, responsive system that makes complex food waste analytics accessible to kitchen managers.

What we learned

The Power of Structured Schemas: We learned that enforcing a strict JSON response schema is essential for transforming LLM outputs into machine-readable, production-ready data.
The Value of Full-Stack Integration: We realized how combining a Node.js backend with a React UI is critical for turning complex AI-driven vision analysis into user-friendly business intelligence. AI for Domain-Specific Solutions: We gained experience in applying computer vision to solve specific environmental challenges, demonstrating that AI can significantly reduce food waste when combined with targeted nutritional intelligence.

What's next for Cafeterian Lens

Enhanced Trend Visualization: We plan to develop more advanced dashboard charts to track food waste trends across different days, weeks, and seasons. Expanded Multi-Modal Training: We aim to refine our model to recognize a broader range of complex, international, or specialized food items to improve accuracy across various cafeteria environments. Automated Predictive Alerting: We look to implement threshold-based alerts that notify kitchen managers when specific food items consistently exceed target waste percentages.