Rounds AI (Team #21: Loud Labs)

Rounds AI is a shipped iOS app that translates doctor conversations for caregivers using on-device speech recognition and GPT-4o-mini. For the Continual Learning Hackathon, we added an autonomous learning layer: after each transcript analysis, the agent autonomously researches drug interactions and medical context via You.com's Search API, stores findings in a persistent patient knowledge graph, and triggers follow-up reminders and alerts via Composio — all without human intervention. A FastAPI backend deployed on Render serves a live web dashboard showing the agent's brain state, knowledge growth, and learning progression in real-time. Every visit makes the agent smarter. We need Render to host this backend so judges can inspect the agent's brain via URL.

Inspiration

When your loved one is in the ICU, you're drowning in medical jargon. Doctors speak in acronyms, vitals change daily, and you can't Google fast enough to keep up. Caregivers need a translator — but a static translator isn't enough. Medical knowledge evolves, drug interactions matter, and every patient's journey is unique.

My dad received a DOUBLE lung transplant last fall, and I WAS this caregiver. I am creating this for other caregivers, who feel like the ICU is speaking a foreign language during daily rounds. Rounds AI let's the caregiver be a fierce advocate for their beloved patient, during a time when they are unable to advocate for themself.

What it does# Rounds AI — Continual Learning Medical Agent

Rounds AI is a shipped iOS app (submitted to the App Store this week) that records doctor visits using on-device speech recognition and translates complex medical conversations into plain language a caregiver can understand.

For this hackathon, we added a continual learning layer that transforms Rounds from a translator into an autonomous medical research agent:

1. LISTENS — Records and transcribes doctor conversations (Apple Speech, on-device)
2. ANALYZES — GPT-4o-mini processes the transcript with full patient memory context, tracking vital trends from Day 1 baseline
3. RESEARCHES — Autonomously searches real-time medical sources for drug interactions, side effects, and latest research on mentioned conditions (You.com Search API)
4. LEARNS — Stores new medical facts in a persistent patient knowledge graph that grows smarter every session
5. ACTS — Creates follow-up appointment reminders, medication alerts, and trend warnings without the caregiver lifting a finger (Composio)
6. VISUALIZES — A live web dashboard shows the agent's brain state, knowledge growth, and learning progression over time (deployed on Render)

Every visit makes the agent smarter. That's continual learning for healthcare.

How It Meets the Challenge Criteria

• Autonomous operation: After hitting "record," the agent handles research, learning, and action creation with zero human prompting
• Continual learning: Patient knowledge graph persists and grows across sessions — Day 10 analysis is dramatically richer than Day 1
• Real-time data: You.com Search API grounds every analysis in current medical knowledge, not static training data
• Meaningful action: Composio triggers calendar reminders, medication alerts, and trend warnings automatically
• Observable intelligence: Web dashboard on Render lets you watch the agent get smarter in real-time

Sponsor Integrations

Render URL: https://rounds-ai-jvse.onrender.com/

Tech Stack

• iOS App: Swift/SwiftUI, Apple Speech Recognition (on-device), GPT-4o-mini
• Backend: Python FastAPI on Render
• AI/ML: OpenAI GPT-4o-mini with custom medical system prompt, You.com Search API for real-time grounding
• Agent Layer: Composio for autonomous actions, custom knowledge graph with vital trend tracking
• Memory: Persistent patient knowledge graph with Day 1 baseline tracking, medical term normalization, cross-session pattern detection

What Makes This Special

This isn't a hackathon prototype — it's a production app that we made smarter today. The continual learning layer we built transforms a shipped product into an autonomous agent that gets better at advocating for patients with every single visit.

Team #21

• Katie Mac — CEO/Founder, Loud Labs (former Head of Developer Programs, Meta)
• Built with: Claude (Anthropic) as technical cofounder

Links

• GitHub: https://github.com/loudkatie/rounds-continual
• Loud Labs: https://loudlabs.xyz
• Render Dash: https://rounds-ai-jvse.onrender.com/

Built With

• composio
• gpt-40-mini
• ios
• openai-gpt-4o-mini
• python
• render
• swift
• youdotcom