CareBridge Ambient OS

24/7 ambient clinical copilot for seniors, powered by NVIDIA Nemotron on Nebius Token Factory & Tavily API for live drug interaction safety.

Comment

💡 Inspiration: From a Summer in Ea Kar to a Distant IT Job — The Evolution of CareBridge

A while ago, when I was still a software engineering student at UIT (University of Information Technology) in Ho Chi Minh City, an astrology website I had poured six months of heart and soul into collapsed overnight following a major search algorithm update. Burned out, financially depleted, and questioning my path, I packed my backpack and took the long bus ride back to my hometown in Ea Kar, Dak Lak—a quiet farming district in the Central Highlands of Vietnam—to stay with my grandmother.

Living with her for several months opened my eyes to an everyday, silent struggle that never crosses our minds in bustling tech hubs.

My grandma lives with diabetes and severe hypertension, taking 6 to 8 different prescription pills every single day. Her eyesight is fading with age, and her fingers tremble with arthritic pain. Every morning, I watched her hold blister packs up to the sunbeams filtering through the window, squinting to decipher tiny pharmaceutical labels, terrified of double-dosing or taking the wrong pill. Whenever a tiny white tablet slipped from her trembling fingers and rolled across the tiled floor, she would apologize to me—as if growing old were her fault. I built the very first prototype of CareBridge back then as a mobile app to help her manage her regimen. While I lived under the same roof, it brought her comfort.

Fast forward to today: I graduated from UIT and am now working full-time as an IT software engineer in Ho Chi Minh City.

Living hundreds of miles away from Ea Kar, a painful new reality set in. At the office, staring at lines of code, my mind constantly drifted back to that quiet house:

Did grandma remember her morning blood pressure pill? Did she feel dizzy today? Why hasn't she picked up my call?

That was when I confronted the harsh truth about mobile healthcare apps for the elderly: A smartphone only works if the senior keeps it with them.

Back home in the countryside, elderly adults don't carry smartphones in their pockets. My grandma leaves hers inside a jacket pocket in another room, forgets to charge it for days, or simply cannot hear the ringtone over the kitchen noise. Whenever the phone rings, she has to shuffle across the house with stiff joints, squinting at an unlocking screen.

Sitting at my engineering desk, I asked myself:

Why are we forcing an 78-year-old grandmother to adapt to smartphones and proprietary app stores, when ambient technology should adapt to her living environment?

She didn't need another screen to tap. She needed Ambient Computing powered by Sovereign, Open AI—an always-on, always-attentive bedside station plugged into the wall 24/7. A glanceable smart display where she doesn't need reading glasses or motor dexterity. She just wakes up, glances across the room at high-contrast numerals, or simply speaks into the air:

"CareBridge, what pills do I take today?"

And as an engineer, I also realized closed, proprietary big-tech ecosystems lock families into closed gardens. Healthcare data demands open, privacy-respecting, high-performance infrastructure. That is why when the Nebius x NVIDIA Global AI Hackathon was announced, I rebuilt CareBridge from the ground up on open weights: powered by NVIDIA Nemotron on Nebius Token Factory, grounded with live Tavily Search API medical verification, and orchestrated through open Model Context Protocol (MCP) standards.

This is the continuation of my promise to my grandmother—bridging the miles between an IT office in the city and a quiet bedroom in Ea Kar.

🌟 What It Does

CareBridge Ambient OS transforms ambient displays into an autonomous, 24/7 proactive bedside healthcare copilot for seniors and remote families:

  • 6-Foot Glanceable Bedside Display (10-Foot UI): High-contrast typography (WCAG AAA compliant), oversized 56px+ tremor-tolerant touch targets ("I TOOK MY PILL"), and a circadian-friendly dark ambient mode engineered for aging eyes across the bedroom.
  • Dual-Tier NVIDIA Nemotron Intelligence:
    • ⚡ FAST Tier (Nemotron-3-Nano-30B): Handles everyday conversational check-ins, routine dose confirmations, and adherence tracking with sub-350ms Time to First Token (TTFT).
    • 🧠 ULTRA Tier (Nemotron-3-Ultra-550B): Automatically engages for acute symptom triage, chest pain assessments, and complex polypharmacy contraindications.
  • $3,000 Tavily Award Engine — Live Clinical Web Grounding: Dynamically queries Tavily Search API (https://api.tavily.com/search) against live FDA black-box warnings and clinical trials, cross-verifying queries (e.g., "Can I take Warfarin with my daily Baby Aspirin?") against our 15-drug Beers Criteria geriatric registry.
  • Web Audio Reactive NVIDIA Green Ambient Glow (#76B900 / #10B981): Fluid Web Audio API AudioContext and AnalyserNode reactive light halo that pulses rhythmically to real-time voice cadence and synthesized guidance.
  • Full Tri-Pillar Model Context Protocol (MCP) Server: Complete compliance with Anthropic MCP specifications, implementing 7 Tools, 2 Resources, and 2 Prompts over Streamable HTTP (SSE).
  • Autonomous Smart Pharmacy Refill Hub: Proactively monitors medicine inventory upon dose logging. When remaining pills drop to $\le 5$ tablets, it coordinates automated 1-click replenishment with express delivery tracking.
  • IoT Porch Camera & Paramedic Smart Deadbolt Unlock: Simulated 850nm IR front porch camera feed with computer vision parcel delivery detection, paired with a one-tap emergency smart deadbolt release for paramedics during acute triage.
  • Hospital-Grade A4 Doctor Summary & FHIR QR Code: One-tap clinical summary sheet featuring a 30-day blood pressure longitudinal curve, eMAR compliance table, and a scannable HL7/FHIR compliant QR code for hospital visits.

🏗️ How We Built It

CareBridge is architected as an enterprise-grade full-stack TypeScript monorepo combining open AI models, standard protocols, and resilient cloud edge deployment:

1. NVIDIA Nemotron Engine on Nebius Token Factory

  • Powered by nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B and nvidia/Nemotron-3-Ultra-550b-a55b served via high-throughput Nebius Token Factory endpoints (https://api.tokenfactory.nebius.com/v1).
  • Utilizes standard OpenAI-compatible SDK integrations with dynamic model tier routing (selectNemotronModelTier).
  • Enforces dual clinical safety guardrails: Topic Denial (blocking unauthorized self-directed cardiac dose adjustments) and Sensitive PII Redaction (masking SSN, credit cards).

2. Live Drug Safety Grounding via Tavily Search API

  • Orchestrates real-time web retrieval (searchTavilyDrugInteraction) with prompt engineering tailored to geriatric pharmacology ("FDA drug interaction [Med1] and [Med2] geriatric").
  • Integrates seamlessly with the 2023 American Geriatrics Society (AGS) Beers Criteria® registry.

3. Tri-Pillar Model Context Protocol (MCP) Server

  • Implemented using @modelcontextprotocol/sdk and Express with Streamable HTTP (SSE) at /sse and /message.
  • 7 MCP Action Tools: getTodaySchedule, logDoseStatus, recordVitals, clinicalAdvisor, orderRefill, ringDeviceHub, negotiateAdherence.
  • 2 MCP Resources: carebridge://patient/eleanor-vance/adherence-30d, carebridge://clinical/prescriptions/active.
  • 2 MCP Prompts: morning_medication_checkin, acute_chest_pain_triage.

4. Hardware-Fidelity Ambient Frontend (Next.js 15)

  • Built with Next.js 15 (App Router), React 19, Tailwind CSS, and Recharts.
  • Direct browser Web Speech API and Web Audio API integration for hands-free senior voice interaction.

5. Resilient Data Storage & Multi-Cloud Edge Deployment

  • Local persistent storage powered by better-sqlite3 in Write-Ahead Logging (WAL) mode with automated demo seeding (usr_demo).
  • Frontend: Hosted on Vercel with global edge CDN and mandatory HTTPS for secure browser microphone permissions.
  • Backend: Hosted on Render supporting persistent long-lived SSE connections and instant REST endpoints.

🧗 Challenges We Ran Into

  1. Nebius Token Factory BaseURL Normalization: During initial OpenAI SDK configuration, passing endpoint variants without /v1 or trailing slash discrepancies triggered routing issues. We resolved this by building strict URL normalization in our environment loader and documenting the behavior in our Developer Friction Log.
  2. OpenAI Spec Tool-Calling & JSON Serialization on Nemotron-3-Nano: When streaming structured arguments under fast response times, small token delimiters occasionally required robust deserialization. We engineered parseJsonSafe with deterministic heuristic clinical fallbacks to ensure the bedside display never stalls.
  3. Dual-AI Latency Budgeting (Nemotron + Tavily): Chaining deep LLM clinical reasoning with live external web retrieval via Tavily required managing network latency. We tuned the frontend client with a resilient 25,000ms abort budget (DEFAULT_TIMEOUT_MS) to ensure complex queries complete reliably without premature client aborts.
  4. Smart Display Acoustic Echo Feedback: On open bedside speakers, the microphone would occasionally capture the agent’s own synthesized voice. We implemented an audio mutex state (isBusyRef) with mutual-exclusion locks tied strictly to speechSynthesis.onend audio events.
  5. Geriatric Medication Refusal (“Sarah Connor Circuit-Breaker”): Seniors frequently resist medication due to nausea, confusion, or mood. We engineered a multi-turn empathetic negotiation tool (negotiateAdherence). If resistance persists beyond 2 consecutive turns, the system engages a safety circuit breaker, notifying remote family without creating confrontation for the senior.

🏆 Accomplishments That We're Proud Of

  • 100% Passing Automated Test Suite: 70 automated Vitest unit and integration tests passing across 8 test suites, validating everything from MCP primitives to Nemotron guardrails and multi-tenant SQLite WAL isolation.
  • Sub-350ms Time to First Token (TTFT): Leveraging Nebius Token Factory and Nemotron-3-Nano to deliver near-instant voice feedback and status transitions.
  • Truly Open Infrastructure: Moving away from closed, proprietary smart-home silos to open-weights NVIDIA Nemotron models running on open, independent GPU infrastructure that healthcare organizations can audit and control.
  • Dual-AI Clinical Grounding: Successfully uniting the 2023 AGS Beers Criteria with live Tavily web intelligence to catch real-world lethal drug combinations like Warfarin and Aspirin.
  • Comprehensive 10-Entry Developer Friction Log: Compiled FRICTION_LOG.md documenting authentic Developer Experience (DX) insights to help Nebius and NVIDIA platform teams refine their next-generation developer tooling.

📚 What We Learned

  • Open Infrastructure is Healthcare Sovereignty: Geriatric health telemetry and medication compliance data are deeply intimate. Serving models on independent infrastructure like Nebius Token Factory guarantees data privacy without sacrificing state-of-the-art reasoning.
  • Cascading Model Architectures are Essential: Routine daily checks do not need massive 500B+ models. Routing everyday queries to Nemotron-3-Nano keeps ambient smart displays ultra-responsive while conserving compute budgets for when Nemotron-3-Ultra is truly needed.
  • Ambient Computing is Compassion in Code: True accessibility for the elderly isn't about larger buttons; it's about removing the device hunt altogether. Meeting seniors in their natural living environment restores dignity to aging.

🚀 What's Next for CareBridge Ambient OS

  • Edge Deployment on NVIDIA Jetson Orin Nano: Moving the Nemotron-3-Nano inference pipeline directly onto on-premise NVIDIA Jetson Orin Nano edge hardware, enabling 100% offline, zero-latency clinical triage in rural homes with intermittent internet connectivity.
  • Physical Smart Pill Dispenser Integration: Coordinating with IoT hardware carousels to physically dispense the correct pill compartments upon voice confirmation.
  • Rural Clinic EHR Telehealth Sync: Connecting the HL7/FHIR QR engine directly into local district hospital EHR systems across rural Vietnam, bridging the gap between home compliance and clinical consultations.

Built With

Share this project:

Updates

Submission history