Itadaki Health

Inspiration

The hackathon theme is patient agency: patients owning and transporting their health data, then receiving more personalized care from AI based on their own context.

That breaks if the AI only sees labs and visit summaries. Meals are part of the patient's health story, but they usually disappear into memory. Cal AI proved that photo-based food logging is a familiar habit. Itadaki turns that habit into patient-directed health data for wearables.

The human story is simple: a lot of families, including Carl's Filipino community, carry health worries like blood pressure, fatty liver, LDL, diabetes risk, and kidney disease as vague background anxiety. Itadaki does not scold the meal that is already on the table. It helps the patient create a record they own, then transport that context into Health Passport, FHIR, or a future care conversation.

What It Does

Itadaki Health is a patient-directed meal memory flow for Meta Ray-Bans and iPhone.

1. The user says "itadakimasu" or taps the low-power capture control.
2. The iOS companion app uses Meta's Device Access Toolkit to capture a Ray-Ban camera frame.
3. The app crops toward the food, sends the image to Grok, and receives structured nutrition estimates.
4. The glasses stay blank until the result lands, then pulse calories and macros for about three seconds.
5. xAI Text-to-Speech plays a short trend line such as "Your recent meals look steady so far."
6. The meal appears as a card in the phone web app and exports to CSV, JSONL, FHIR R4, and Health Passport markdown.

The punchline: Cal AI helps users track food. Itadaki helps patients own and transport food context for autonomous care.

How We Built It

The public demo is a Next.js app on Vercel. The wearable HUD is a 600x600 static Meta Display Web App. The real capture path is a SwiftUI iOS companion app using Meta Wearables DAT, because the Web App display path does not provide the camera and microphone capture path we needed.

The backend has three core routes:

• /api/analyze-meal sends the meal photo and scenario context to xAI for structured food analysis.
• /api/speak generates a short MP3 confirmation with xAI Text-to-Speech.
• /api/log-meal saves the log to JSONL and CSV for the demo, then exposes it through /logs, /api/logs, and /api/health-passport.

Michelle worked on the FHIR lane: recent meals become FHIR-friendly Observations and a lightweight CarePlan shape for trend coaching. That makes the data portable instead of trapped inside another wellness app. The project uses synthetic data only.

Challenges

The main constraint was the Meta split between Web Apps and DAT. Web Apps are excellent for a tiny display HUD, but camera capture needed the native iOS DAT path. That pushed us toward a two-surface architecture: iPhone for capture, glasses for glanceable feedback.

We also had to keep the output calm. A face display is not a dashboard. The final HUD is intentionally blank most of the time, then flashes only the estimate the user needs.

What We Learned

The strongest product is not "can I eat this?" The meal is usually already here. The stronger product is "help me remember what happened, then let me use that memory to direct care later." That is the patient-agency wedge.

What Is Next

The next version connects patient-directed health record import, such as a HealthEx-style flow, so labs, medications, notes, and meal logs can live in one Health Passport timeline. The long-term direction is local-first and patient-owned: cloud for the hackathon, on-device models when hardware allows.

Built With

• avfoundation
• csv
• fhir-r4
• framer-motion
• health
• inngest
• jsonl
• meta-ray-ban-display-web-apps
• meta-wearables-device-access-toolkit
• next.js
• react
• swiftui
• typescript
• vercel
• xai-grok-vision
• xai-speech-to-text
• xai-text-to-speech