ProperFood

ProperFood uses AI to scan restaurant reviews for hygiene, scams, and dietary risks, then gives a transparent safety score, so you can dine with confidence.

Comment

Inspiration

We built ProperFood after a frustrating realization: star ratings lie.

A 4.8-star restaurant might hide hygiene complaints buried in low-star reviews. A traveler with a nut allergy could order at a place with undisclosed cross-contamination risks. A student searching for halal ramen near Hougang might be recommended options in distant neighborhoods with no relevance to their query.

The problem crystallized when we realized that trust in food safety isn't about aggregated scores—it's about auditing what actual customers complain about. We wanted to flip the narrative: instead of asking "Is this restaurant highly rated?", we'd ask "What are customers actually unhappy about, and does it affect me?"

What It Does

ProperFood is an AI-powered restaurant safety recommender that audits real reviews to surface genuine risks.

The workflow:

  1. Parse your intent — Natural language input ("halal ramen near Hougang, no peanuts") → structured preferences (cuisine, dish, location, allergies)
  2. Find candidates — Google Places API search with allergies as filters, defaults to 500m radius if no location given
  3. Filter operationally — Remove closed places, low-rated outliers, price mismatches
  4. Scrape reviews — Apify fetches 8 low-star reviews per candidate (Google Maps scraping)
  5. Audit for risks — Qwen LLM scans reviews for hygiene complaints, scams, dietary violations
  6. Score transparently — Deterministic ranking: Bayesian-adjusted rating + safety penalties + intent bonus + distance bias
  7. Pause for approval — Human-in-the-loop: you select from ranked candidates, system confirms it's safe
  8. Explain the verdict — Qwen generates why this restaurant is safe for you, with score breakdown

Result: A single restaurant recommendation in 15-30s, with transparent scoring and confidence-weighted confidence.

How We Built It

Backend (Python 3.11 + FastAPI + LangGraph):

  • 8-node state machine with SQLite checkpointing (supports pause/resume/cancel)
  • Qwen LLM (DashScope) for preference parsing, review auditing, final explanation
  • Google Places API: Text Search (websiteUri + googleMapsUri in single call), fallback Place Details for shortlist
  • Apify google-maps-reviews-scraper: configured for low-star reviews (8 per place, fallback to 3-4 star if sparse)
  • WebSocket /voice/asr endpoint for real-time speech-to-text (browser AudioWorklet PCM 16kHz)
  • FastAPI Endpoints:
    • POST /search: Starts graph, returns thread_id + ranked candidates (pauses at HITL)
    • POST /approve: Resumes graph with user's selection, returns FinalRecommendation
    • POST /cancel: Signals cancellation via thread-level registry

Frontend (Next.js 16 + TypeScript):

  • Natural language search input + voice capture (browser AudioWorklet)
  • Results page: ranked candidates with scores, HITL pause/resume, final report
  • Profile: save dietary restrictions, allergies, search history
  • Favorites & search history management

Infrastructure (Alibaba Cloud ECS):

  • Docker Compose: FastAPI backend (port 8000), Next.js frontend (3000), Nginx reverse proxy (80)
  • SQLite for LangGraph state persistence (thread_id → state)
  • Tailscale HTTPS for secure microphone access
  • Concurrent workers: 3-thread ThreadPoolExecutor for review fetching & auditing

Caching (3 levels):

  • Review corpus by place_id
  • Audit results by review corpus hash
  • Dish-term expansions by normalized term

Challenges We Ran Into

Sparse Low-Star Reviews Google Places max-returns 5 reviews per query. If candidates lack 1-2 star reviews, no audit data exists.

  • Solved: Adjusted to 3-4 star reviews when low-star data absent; fallback logic relaxes rating filters if zero results

Node 5 Processing Latency Each Qwen audit takes 2-5s; with 5 candidates, that's 10-25s total.

  • Solved: Cached audit results; ran Node 4 & 5 concurrently with 3-worker ThreadPoolExecutor; added UX feedback ("Auditing reviews...")

API Cost Explosion Scraping reviews + Qwen calls + place details adds up fast.

  • Solved: Fetched websiteUri + googleMapsUri in single Text Search call (no extra Place Details per result); only scraped menus for top K candidates; 3-level caching

Dish Intent vs. Metadata Mismatch Menu items like "hojicha" don't appear in cafe metadata. Hard filtering eliminated all results (~20% recall).

  • Solved: Removed hard filtering in Node 3; deferred to soft scoring with dish-term expansion (intent match bonus); recall improved to ~80%

Cancellation Mid-Pipeline Users wanted to abort slow Qwen/Apify calls.

  • Solved: Built thread-level cancellation registry; raised CancelledRunError in concurrent workers; added POST /cancel endpoint

WebSocket Proxying Through Nginx Voice audio streams required proper HTTP upgrade headers and timeouts.

  • Solved: Added proxy_set_header Upgrade $http_upgrade + Connection $connection_upgrade; raised API timeouts to 300s

Distance Bias If user says "Clementi," hard-rejecting nearby Bukit Timah is wrong; but distance shouldn't dominate either.

  • Solved: Added distance bonus in scoring (+0.35 max); sorts by (distance_bonus + rating + popularity), not distance alone

Accomplishments That We're Proud Of

✅ 8-node LangGraph agent with HITL checkpoint + SQLite state persistence
✅ Allergies passed to Places API to narrow search upstream (cost optimization)
✅ Default location handling (500m GPS radius if not specified)
✅ Strict typing via Candidate, UserPreferences, ReviewAudit models
✅ Cost-optimized API flow (websiteUri in Text Search; Place Details fallback only)
✅ Soft dish matching instead of hard filtering (recall: 20% → 80%)
✅ Bayesian confidence adjustment for ratings with low review counts
✅ Distance as bias, not gate (nearer preferred, distant not eliminated)
✅ Concurrent review scraping & auditing (3-worker ThreadPoolExecutor)
✅ 3-level caching (review corpus, audits, dish terms)
✅ Cancellation support via thread-level registry
✅ Voice search via WebSocket PCM streaming
✅ Transparent 9-factor scoring with breakdown
✅ Alibaba Cloud ECS deployment (Docker Compose, Tailscale HTTPS)

What We Learned

  1. Default location handling saves API costs — Defaulting to GPS + 500m radius reduces bloat and improves local relevance

  2. Filter allergies at the source — Passing allergies to Places API narrows results upstream; one fewer API call per result

  3. Strict typing prevents silent failures — Type mismatch catching in multi-stage pipelines is a force multiplier

  4. Soft scoring beats hard elimination — Menu items aren't place attributes; deferring dish matching to scoring improved recall from 20% to 80%

  5. Website URI optimization is the unsexy win — Fetching websiteUri + googleMapsUri in same Text Search call avoids extra Place Details requests

  6. Bayesian priors regularize outliers — A 5.0 with 2 reviews shouldn't outrank 4.2 with 200; confidence matters as much as score

  7. Human-in-the-loop + checkpointing scales — SQLite state persistence + HITL pause enables conversational flow without re-querying expensive APIs

  8. Distance is a bias, not a gate — Users want control over locality without hard constraints; scoring bonuses work better than filters

What's Next for ProperFood

  • Multi-cuisine expansion — Currently tuned for Asian restaurants; generalize to pizzerias, cafes, fusion concepts
  • Allergy risk scoring — Quantify cross-contamination risk per allergen type; surface kitchen hygiene certifications
  • Community flagging — Users upvote/downvote audit findings ("This hygiene complaint was real" / "False alarm") to improve future audits
  • Menu integration — Scrape restaurant websites for full menus; enable "Find restaurants with dish X near location Y" queries
  • Dietary certification verification — Cross-reference user allergies against posted certifications (halal, vegan, gluten-free labels)
  • Real-time review monitoring — Alert users when new low-star reviews appear for their saved favorites
  • Regional deployment — Expand from Singapore to SE Asia (adapt language to local review patterns, regional cuisines)
  • Voice feedback loop — Users voice corrections mid-search ("Actually, I prefer spicy"); retrain scoring weights from feedback

Built With

Share this project:

Updates