Soma | Devpost

Inspiration

Modern healthcare has a fatal flaw: the silence. You see a doctor for 15 minutes once a year, or perhaps only when something is already broken. But your health is defined by the micro-decisions you make during the other 525,000 minutes of the year, like what you eat, how you move, and how you sleep.

During these gaps, patients are blind. We noticed that students and patients alike are drowning in data but starving for wisdom. Your watch counts steps, your portal holds PDF records, and your dining hall has a menu, but none of them talk to each other.

We built this project to end that silence. We set out to eliminate the "healthcare gap" by building a system that doesn't just track you, but cares for you. Our mission was to create agents so capable and aware that it effectively eliminates the need for weekly checkups, managing your daily wellness routine so doctors only need to intervene when it truly matters.

What it does

Our platform is a continuous, autonomous care ecosystem. It unifies your fragmented digital life, consisting of EHR records, Apple HealthKit biometrics, and lifestyle data, into a single "living profile" that powers a team of specialized AI agents.

Bridging the Gap: It acts as a 24/7 care team. Instead of waiting for a checkup to be told your BMI is trending up or your sleep is affecting your medication, our system detects these patterns in real-time and adjusts your plan immediately.

Hyper-Personalized Agents: We deployed a squad of specialized agents named Nutri (Diet), Luna (Sleep), Rex (Fitness), and Soma (General). They don't give generic advice. They know you have a gluten allergy, they know you only slept 4 hours last week, and they know you prefer lifting over cardio, and automatically adjusts the daily checklist of meals, exercise, sleep, and tasks based on that.

Adaptive Daily Plans: Life happens. If you wake up tired or have a busy exam schedule, the system modifies your Daily Checklist instantly, adapting to conveniences and needs.

The Nutrition Solution (UCSB & Beyond): We solved the "what do I eat?" problem. We scraped UCSB Dining Hall menus so students can get exact meal recommendations like "Get the grilled chicken at Carrillo" that hit their macro goals. For everything else, we built a Food Scanner. Snap a picture of any meal, anywhere, and our AI identifies the ingredients and calculates the nutrition facts instantly, logging it to your profile.

How we built it

We built an edge-first IoT sensor fusion layer using React Native + Expo and Apple HealthKit, where the phone acts as the edge device aggregating high-frequency biometrics (steps, heart rate, sleep, workouts, BMI) with seamless sync, accessibility, and offline-first behavior. A resilient, event-driven data pipeline implements idempotent per-user-day upserts into normalized Supabase Postgres schemas, protected by Row Level Security for strict tenant isolation and PII obfuscation; we batch and cache requests to meet latency budgets and prevent rate-limiting at Vercel scale.

On the intelligence side, a Vercel-hosted multi-agent mixture-of-experts (Nutri, Luna, Rex, Soma) uses a lightweight Router with reasoning budgets, confidence scoring, and context window optimization to compile only relevant EHR vectors and recent biometrics. Our custom EHR NER parser performs domain-specific entity extraction and normalization from noisy, heterogeneous clinical JSON, while the multimodal Food Scanner applies Visual Chain-of-Thought and cross-modal alignment to turn pixels into nutrition macros; the UCSB Dining Scraper mines unstructured web data into normalized datasets, boosting signal-to-noise for recommendations and anomaly detection.

The system forms a cybernetic feedback loop: daily actions (meals, workouts, sleep) update state and drive Root Cause Analysis for the human body—surfacing upstream issues (poor sleep) behind downstream anomalies (elevated resting HR). With clean, type-safe APIs and modular components, Responsible AI guardrails, server-side key management, and user-centric mobile UX, the platform delivers fast, explainable guidance that is reproducible, secure, and production-ready.

Challenges we ran into

HealthKit Integration: Navigating Apple's strict privacy policies and permission granularity was complex. We had to build robust error handling for scenarios where users granted "Steps" access but denied "Heart Rate," ensuring the app didn't crash on partial data.

Context Retrieval: Early iterations of our agents struggled to "remember" specific medical conditions when the context window became crowded with daily biometric logs. We solved this by implementing a dynamic context compiler that prioritizes clinically relevant data (like active medications) over noisy sensor trends for medical queries.

Accomplishments that we're proud of

Closing the Loop with Multi-Agent AI: We didn't just build a chatbot; we engineered a team of specialized agents that actually coordinate. Seeing the "Nutri" agent recommend a specific high-protein meal from the UCSB dining hall because the "Rex" (Fitness) agent logged a heavy lifting session was a breakthrough moment in proving true context-aware health management.

Reverse-Engineering the Dining Experience: We successfully built a scraper that turns static UCSB dining menus into a queryable nutrition database. Combining this with our multimodal food scanner means we effectively solved the hardest problem in student health: knowing exactly what is on your plate, whether it's from the dining commons or a home-cooked meal.

What we learned

We learned that true healthcare is proactive, not reactive. By placing powerful, context-aware AI in a user's pocket, we realized we could shift the paradigm from "treating sickness" to "optimizing life."

What's next for Soma

We plan to implement full FHIR (Fast Healthcare Interoperability Resources) support, allowing our system to not just ingest records, but securely write daily health summaries back to your doctor’s clinical dashboard.