AEGIS

patient overview & chatbot configuration
chatbot for patient education
daily calendar

Inspiration

AEGIS was inspired by the challenges that both patients and healthcare providers face. Patients often struggle with access to care, understanding medical advice, or even feeling heard, especially in multiple languages. On the other hand, doctors are overwhelmed with time constraints and administrative tasks, making it difficult to deliver personalized care efficiently. We wanted to bridge that gap by building a tool that could make medical conversations more accessible, accurate, and actionable.

What it does

AEGIS captures real-time conversations between patients and doctors using Whisper for speech-to-text transcription. It then processes and stores the transcript in a PostgreSQL database. With the help of the Claude API, it recalls the care plan and documents that a doctor configures the chatbot with, enabling patients to ask targeted questions without having to read through a stack of papers.

How we built it

We used the following technologies:

Whisper for real-time transcription of doctor-patient conversations
Claude API for natural language understanding and summarization
FastAPI to manage backend logic and API endpoints
PostgreSQL to store and retrieve transcripts and summaries

The system streams audio input to Whisper, stores the resulting transcript in the database, and uses Claude to generate intelligent responses based on the transcripts+online medical links+educational documents. The chatbot interacts with this information in real time, enabling users to revisit important points or ask questions about their care.

Challenges we ran into

-Narrowing our scope: We had to scale back our ambitious ideas and focus on the most impactful features first.

Integrating tools: Whisper, Claude, FastAPI, and PostgreSQL each had their own quirks. Making them work smoothly together took time and trial-and-error.
Real-time syncing: Getting Whisper to transcribe live audio and immediately feed that into our backend for processing and chatbot use was particularly difficult.

Accomplishments that we're proud of

Real-time conversation streaming and automatic checklisting for points a doctor wants to bring up using Whisper+Claude. Very narrowly scoped chatbots that cite documents and trusted inks doctors want to give their patients, replacing the traditional flow of doctors giving their patients a stack of papers they will never read, especially post-operation.