The Problem

A main source of admin burden in healthcare is that the patient intake forms act as a bottleneck, holding up hospital visits until the paperwork is completed. Traditional methods such as having patients fill out the paperwork themselves or requesting medical staff complete it for them are slower, have more room for human error, and force the communication of information in unnatural and unintuitive ways.

What It Does

Intake collects information from the patient, automatically fills out the intake forms for them, and flags areas where further verification is recommended. It speeds up the process while reducing human error by gathering information through a conversational format with the patient. Intake uses AI agents to avoid burdening hospital staff while increasing accessibility by allowing for typed or spoken responses.

How We Built It

We started by looking at forms that are given to patients at real clinics, and extracted all the questions that were asked. Then, we created a split-screen visualisation with a chatbot on the right, and the dynamically-updating form on the left. As the user talks to the chatbot (either with voice or text), it extracts the data into a JSON and produces a confidence score based on how well the answer matches the questions. If the user answers multiple questions at once (eg. “I hurt my knee playing ping pong” answers the cause for visit and whether or not the injury is sports-related), the chatbot can recognize that and fill in both at once. The chatbot is programmed to have a casual and free-flowing conversation with the user, and when the conversation is finished, it outputs a summary and flags all questions as either complete or needing review.

To make this all happen, we used Arize Phoenix to build and debug the agent (debug terminal runs at all time and gives error messages), LiveKit Cloud for the voice recognition, Claude Haiku 4.5 as our model for maximum speed, and a modern full stack including React and Tailwind for the frontend combined with FastAPI for the backend.

What we learned

Prompting is actually really important for applications like this: good prompting allows the LLM to update multiple fields at once, know which answers to accept, ask for clarification when answers are too vague, and keep the conversation casual and free-flowing.

What's next for Intake

We’d love to add more features to Intake, including token optimization to improve efficiency or adding a camera that looks at the user and sends their facial expressions (worried, angry, etc) as data. We think something like Intake could be used in hospitals around the world to make life easier for hospital administrators and patients alike.

Built With

  • arize
  • claude
  • livekit
  • overshoot
  • python
Share this project:

Updates