Flanner

Home page
Cart page
Knot asked the inventory from real store
K2 makes 7days meal plan
Handle text iMessage using Photon
Handle multimodal iMessage using Photon

Inspiration

Delivery food carries 2 to 3 times the sodium and roughly 40% more calories than home-cooked meals. The CDC reports that 9 in 10 U.S. adults over-consume sodium, and adult obesity has crossed 42%, with restaurant and takeout food flagged as a leading driver of hypertension and metabolic disease.

We felt this ourselves. Between DoorDash, Uber Eats, and Grubhub, our ordering habits were completely fragmented, and no single app could show us what six months of takeout actually looked like. To make the problem concrete, we seeded a realistic 6-month history from Knot's development-mode data and extended it into a believable demo persona: 287 orders, 42 of them bubble teas, daily sodium blowing past the CDC's 2,300 mg guideline every single day. Watching it rain down the screen at once is a very different experience from any macro dashboard. It is the shift from a number to a mirror.

The core problem: habits stay invisible while the data is fragmented. Flanner unifies delivery history across platforms, surfaces what you have actually been eating, and automatically designs a weekly meal plan of home-cooked alternatives that taste like the food you already love, but with clinically meaningful reductions in sodium and calories.

What it does

Flanner uses Knot TransactionLink to unify order history from DoorDash, Uber Eats, and Grubhub, then runs an AI pipeline to design a healthier week:

Delivery habit analysis. Visualizes your most-ordered foods and estimated sodium / calorie intake over 6 months, benchmarked against CDC and AHA guidelines.
Flavor-matched healthy recipes. K2 Think V2 maps each frequently ordered dish to a flavor-similar home-cooked alternative with lower sodium and fewer calories (Korean BBQ → bulgogi bowl, shrimp tacos → home tacos).
Automated grocery cart. Aggregates the week's ingredients, subtracts pantry contents, and pushes the exact basket into your real Amazon Fresh account via Knot AgenticShopping in production mode: real OAuth, real merchant, real ASINs.
Daily check-in. A local Gemma 4 chatbot asks "Did you make the bulgogi today?" over iMessage, tracks adherence, and suggests uses for leftover ingredients. Adherence data never leaves the user's device.
Health impact simulation. Side-by-side monthly report: sodium reduced ~40%, calories ~25% lower, protein quality up, all computed from order data, not hypotheticals.

How we built it

Frontend

Next.js 15 + React 19 (App Router), Tailwind CSS 3, Framer Motion 11
Zustand 5 with persist middleware for onboarding, meal plan, and cart state
Spring-based animations (stiffness 140 to 220, damping 12 to 18, 40 to 80 ms stagger) for a hand-tuned "sticker" UI: 3 px charcoal outlines, peach / mint / lavender palette

Backend

FastAPI + MongoDB Atlas, containerized with Docker and deployed to Google Cloud Run (us-central1, autoscaling 0 to 3)
K2 Think V2 (MBZUAI). Primary planner. Evaluates 7 constraints simultaneously: sodium ≤ 1,500 mg per meal, daily calories ≤ 2,000 kcal, Google Calendar conflict avoidance, pantry reuse, flavor similarity, ingredient overlap minimization, dietary restrictions. After generation, K2 self red-teams its own plan by adopting an adversarial persona and rejecting any output that violates a constraint.
Gemma 4 Vision. Recognizes food photos and returns calories / protein / sodium estimates for receipt OCR and fridge snapshots.
Gemma 4 (local via Ollama). The daily check-in chatbot. Privacy-first by design: dietary-adherence conversations never leave the user's laptop.

Image pipeline

Stable Diffusion XL generates 66 food, ingredient, mascot, and meal assets
rembg for background removal; prompts managed declaratively via prompts.yaml

Integrations

Knot TransactionLink. OAuth-connected delivery history across DoorDash, Uber Eats, Grubhub. The raw signal for nutritional analysis.
Knot AgenticShopping. Automated Amazon Fresh cart loading in production mode. /cart/checkout is hard-wired to simulate=failed, so there is no real-charge code path in the repo by design.
Google Calendar OAuth. Read-only events fed into the K2 prompt so complex dinners do not land on busy weeknights.
Rainforest API. Real Amazon ASIN discovery for groceries.
Photon iMessage kit. Bridges iMessage ↔ FastAPI on a presenter Mac for the daily check-in loop.

Nutritional grounding: USDA FoodData Central for macros; dietary targets anchored to USDA Dietary Guidelines for Americans 2020 to 2025, AHA sodium guidance, and DASH references.

Challenges we ran into

Going from sandbox to production on Knot. Production /cart refuses to run until a real Amazon account is OAuth-linked under the exact external_user_id. Figuring that out required live probes against prod endpoints and scoping which Knot session types are approved for our client ID (transaction_link yes, shopping no on this account). We wrote a separate knot-prod.md runbook just to demo it safely.
Delivery orders into nutritional data. Knot TransactionLink provides payment-level data (item names, restaurants), not nutrition facts. We built a separate pipeline to infer ingredients from menu names, map them to USDA nutritional profiles, and supplement with Gemma Vision where photo evidence was available.
Balancing 7 constraints in the planner. Asking K2 to satisfy flavor similarity + sodium + calories + calendar + pantry + dietary restrictions + ingredient overlap all at once caused one constraint to silently override another. We solved it with a priority chain (K2 → Gemini → deterministic mock) plus a post-validation layer that catches violations before output.
SDXL fp16 on Apple Silicon. The SDXL UNet attention layer produced NaN in fp16 on MPS, resulting in all-black outputs. We added automatic fp32 downgrade detection for Apple Silicon.
Building a believable 6-month history. We pulled a seed batch from Knot's development-mode sample data (287 transactions) and extended it with realistic weekly ordering patterns (bubble-tea dominance, weekday lunch clustering, sodium trends) via a deterministic PRNG, so the demo reproduces exactly the same visualization every time.
Mac-native iMessage. Photon's iMessage bridge cannot be containerized, so we kept the check-in loop on a presenter laptop and sent everything else through HTTPS.

Accomplishments that we're proud of

Production Knot, not sandbox. At demo time, a judge scans a QR into the presenter's actual Amazon Fresh account and watches Flanner push real ingredients in real time: real OAuth, real merchant, real ASINs. Most hackathon integrations stop at a sandbox screenshot; we went all the way.
Safe-by-construction checkout. The absence of a real-checkout function is a feature. Anyone auditing this repo can confirm Flanner literally cannot charge a card. /cart/checkout is hard-wired to simulate=failed, and no simulate=succeeded path exists anywhere in the codebase. In a domain that touches both money and health behavior, this posture matters.
5-stage cinematic onboarding. 287 delivery receipts cascade onto the screen, ingredients burst out of each dish, and a grocery cart fills itself: a 30-second sequence that makes 6 months of habits viscerally tangible. The volume is what triggers the "wait, that's me?" moment.
Mirror Graph. A directed-graph visualization that maps each delivery favorite to its home-cooked counterpart ("my go-to Korean BBQ becomes a 520 kcal bulgogi bowl"), so the health upgrade clicks instantly.
Quantified, guideline-grounded impact. Sodium down ~40%, calories down ~25%, improved protein quality, all benchmarked against CDC / AHA / USDA targets, not aspirational numbers.
On-device daily check-ins. Adherence chats run through local Gemma 4 via Ollama. Dietary behavior, the most sensitive data in the loop, never touches our servers.

What we learned

Payment data can become health data. Delivery transaction history, a financial dataset, can be reverse-engineered into sodium, calorie, and dietary patterns. Knot's TransactionLink turns out to be a powerful data source for preventive-health applications.
Awareness drives behavior change. Showing someone a number is easy to ignore; showing them 287 receipts raining down their screen is not. How you visualize the data is what actually triggers healthier choices, a principle drawn directly from self-monitoring research in behavioral dietetics.
More constraints demand hybrid approaches. Pure LLM reasoning consistently dropped 2 to 3 of our 7 nutritional constraints. Post-validation plus a fallback chain proved essential: judgment is cheap, verification is load-bearing.
Safety is a design posture, not a disclaimer. Flanner is a wellness tool, not a medical device. It does not diagnose, treat, or claim to prevent disease. Users with clinical conditions (hypertension, CKD, diabetes, disordered eating) are prompted to consult their care provider before adopting a plan. Anchoring every recommendation to public guidelines (CDC, AHA, USDA) and removing the real-charge code path are part of the same commitment: the thing cannot hurt you, on purpose.

What's next for Flanner

Princeton campus pilot. A 4-week pre-post study measuring sodium intake (via 24-hour dietary recall) in students who order delivery 3+ times per week, in partnership with a registered dietitian.
Condition-specific modes. Low-sodium (CKD / HTN), low-GI (pre-diabetes), Mediterranean (cardio-protective), each grounded in published dietary patterns.
Auto-scheduled weekly plans. K2 regenerates every Sunday at 8 PM using the past week's adherence signals.
Apple Health integration. Pull activity-level and vitals data so recommendations respond to real energy expenditure, not just delivery patterns.
Receipt OCR into pantry deltas. Finish the UI so users can photograph their fridge weekly and let Flanner reason about leftovers.
Multi-grocer. Whole Foods and Walmart next, once Knot's AgenticShopping expands merchant coverage.
Shared households. The most interesting case is three people with conflicting dietary restrictions and one shared pantry.
Causal analysis. When adherence drops, can K2 explain why? ("You skipped Wednesday's dinner the last four weeks, and your calendar shows a recurring late meeting. Let's make Wednesday simpler.")