Daily Appointment Assistant

Inspiration

During hackathon we are drawn to the fact that time constrained physicians are often time struggle to quickly digest patient histories before rounds. Despite the availability of electronic health records, the data is scattered and time-consuming to parse.

What it does

This companion helps doctors get up to speed before seeing patients by summarizing key information such as conditions, medications, allergies, and visit history. It generates concise, speech-ready summaries that can be reviewed quickly—supporting doctors during prep or while walking between offices.

How we built it

We built a full-stack application with a React frontend and a Python-based backend using FastAPI. The backend integrates with clinical schedule data and a local or remote summarization model to generate daily summaries. For speech output, we cached and stored generated summaries using client-side state and local APIs. Initially, we used a locally deployed medical summarization model, but eventually integrated OpenAI APIs for improved quality and performance.

Challenges we ran into

Offline model deployment took significant effort to configure, and performance for simple models was not satisfactory. Due to latency and quality issues, we switched to OpenAI’s API-based summarization, which offered faster and more coherent outputs. Managing the variability in patient history formats and incomplete data was another technical challenge.

Accomplishments that we're proud of

We successfully integrated a working pipeline that combines clinical data with natural language generation to support real-world healthcare workflows. We’re proud that we built a prototype that applies machine learning techniques in a practical setting and demonstrates potential value in clinical environments.

What we learned

What's next for Daily Appointment Assistant

We plan to expose the assistant’s voice interface to integrate with other clinical automation workflows, such as smart speakers or on-premise kiosks. We’re also exploring deployment of localized models for HIPAA-compliant environments, along with implementing guardrails to reduce hallucinations and ensure summaries remain factually grounded.

Built With

• fastapi
• llm
• react