GrocerySense AI — Project Submission & Development Overview

Inspiration

Deciphering food labels in a grocery aisle is overwhelming. Millions of people suffer from severe food allergies, dietary restrictions, or gut sensitivities, yet ingredient lists are deliberately obfuscated with complex chemical codes (e.g., E120, E250) and obscure technical aliases (casein, sodium caseinate, albumin). Furthermore, individual food items are rarely evaluated in combination—consuming multiple synthetic additives together can create compounding toxicological effects on gut health.

We created GrocerySense AI to democratize food safety. By giving shoppers an instant, voice-enabled AI toxicologist and nutritionist right in their pocket, we empower individuals and families to make safe, informed dietary choices in seconds.

What it does

GrocerySense AI is a state-of-the-art, real-time food safety scanner and dietary health assistant for Android:

• ⚡ Real-Time AI Ingredient & Allergen Scanner: Uses CameraX and Google ML Kit OCR to instantly scan ingredient labels or barcodes. Groq LLaMA 3.3 analyzes the text to flag direct allergies, hidden ingredient aliases, ultra-processed (NOVA 4) foods, and harmful additives.
• 🔊 Voice-Activated AI Verdicts: Synthesizes custom spoken audio warnings using ElevenLabs Cloud TTS (with HD Android system TTS fallback) so users can listen hands-free while shopping.
• 🔥 AI Calorie & Macro Photo Scanner: Uses Groq Vision AI (llama-3.2-11b-vision-instruct / qwen-2.5-32b) to analyze snapshot photos of cooked meals, estimating calories, protein, carbs, fats, and potential allergen risks.
• 🧪 Chemical Lab Toxicologist Simulator: Simulates the combined gut microbiome and organ impact of consuming multiple synthetic dyes and preservatives simultaneously (e.g., E250 + E102).
• 👨‍👩‍👧 Family Scan Mode: Simultaneously cross-references scanned items against multiple family member health profiles in a single scan.
• 🩺 Recipe Doctor & Smart Shopping List: Identifies risky ingredients in home recipes, suggests 100% safe culinary swaps, and adds alternative ingredients straight to an interactive shopping checklist.

How we built it

• Platform & Design System: Built natively for Android using Kotlin, Jetpack Compose, and Material 3 with a custom dark glassmorphism UI theme.
• AI Core & Vision Engine: Powered by Groq LLaMA 3.3 (llama-3.3-70b-versatile) for ultra-low latency text analysis, paired with Groq Vision / Qwen 2.5 for multimodal meal picture analysis.
• Voice Synthesis: Integrated ElevenLabs API (eleven_multilingual_v2) for realistic voice warnings.
• Computer Vision & Hardware Integration: Google ML Kit (Barcode Scanning & Text Recognition OCR) + Android CameraX for real-time frame capturing.
• Local Persistence & Data Source: Room Database and DataStore Preferences for offline profile storage and scan logs, backed by OpenFoodFacts API for barcode lookup.
• Architecture: Strict Clean Architecture (Data, Domain, UI layers), MVVM design pattern, and Hilt for dependency injection.

Challenges we ran into

1. Obfuscated Allergen Aliases & Prompt Engineering: Raw OCR output from crinkled or curved food packaging is often noisy. Prompting LLaMA 3.3 to reliably identify obscure scientific aliases without hallucinating required extensive prompt iteration and strict structured JSON response schema enforcement.
2. Audio Latency & Smooth Playback: Streaming ElevenLabs TTS audio while processing heavy LLM requests simultaneously presented threading challenges. We built a custom manager (OrpheusTtsHelper) with audio queue management and fallback to Android HD TTS to guarantee zero UI stuttering.
3. Multi-Profile Cross-Referencing: Evaluating a single product against N different family profiles simultaneously required optimizing the domain AnalyzeScanUseCase to run parallel evaluations without ballooning token usage or API latency.

Accomplishments that we're proud of

• Near-Instant Analysis: Achieving sub-second AI ingredient analysis using Groq’s high-speed inference engine.
• Multi-Additive Toxicology Simulation: Successfully designing the Chemical Lab Simulator, which models cocktail effects of combined synthetic food additives that standard ingredient lookup apps ignore.
• Fluid UI & Accessibility: Delivering a dark glassmorphic Compose UI complete with smooth Lottie animations, hands-free voice readouts, and zero clutter.

What we learned

• High-speed inference changes UX: Leveraging Groq’s high-throughput hardware allowed us to treat LLMs as near-instant edge-like functions rather than slow asynchronous network calls.
• Multimodal safety auditing: Combining OCR text extractions with Vision AI models provides significantly higher accuracy when dealing with damaged packaging or missing barcodes.
• Designing for dietary accessibility: Food safety isn't one-size-fits-all; multi-user profile matching is crucial for households managing different dietary needs.

What's next for GrocerySense AI

• Offline On-Device AI: Integrating lightweight quantized on-device SLMs (e.g., Gemini Nano / LLaMA 3B) for complete offline functionality in store basements with spotty cellular service.
• Medical & EHR Integration: Allowing users to import dietary restrictions directly from health platforms or doctor prescriptions via FHIR/HL7 standards.
• Microbiome Health Tracking: Expanding the Chemical Lab into a daily gut-health score tracking log over time based on scanned food logs.

Built With

• kotlin