Clara — Your First Aid Copilot
Inspiration
When one of our teammates was six, their younger sister started choking at a restaurant. Their grandparents were trying to help her, speak with 911, and find someone nearby who knew what to do—all at once. By luck, a doctor was nearby, and their sister survived.
We believe that in an emergency, the person providing care should be able to focus on the person in front of them. We built Clara to help manage the instructions, supplies, communication, and search for help around that moment.
What it does
Open the lid, and a limit switch activates Clara’s hands-free voice session. You can speak to Clara through the microphone and hear its responses through the speaker. Clara can help in several ways:
- Instructions: Clara can guide you through a prewritten first aid scenario using spoken instructions and on-screen visuals.
- Equipment: Clara knows what is stocked in each of the box’s three compartments. It can show where an item is and light the corresponding compartment.
- Understanding the situation: Clara can ask questions, inspect an image from its detachable camera when requested, and read a connected pulse sensor. It distinguishes your answers from camera observations and sensor readings; it does not make a medical diagnosis.
- Dispatcher workflow: Clara can continue talking with you while a background task prepares a briefing and calls a team-owned dispatcher phone. If the dispatcher needs information Clara does not have, it can ask you. The current prototype does not dial public 911.
- Volunteer network: Clara can alert nearby people who have opted in as responders or drivers. They can respond through text or the companion app.
The companion app also lets people find a nearby Clara box, check its listed supplies, and sign up to help with future incidents.
Why Clara?
In an emergency, every extra step competes for your attention. Clara brings guidance, supplies, and requests for help into one place:
- Voice-first interaction: You can speak hands-free instead of typing or navigating a small phone screen.
- Parallel tasks: One request can start guidance and background outreach while Clara remains available for conversation.
- Physical context: Clara knows the box’s inventory, can request readings from connected hardware, and can light the compartment containing a needed item.
- One simple interface: The person at the box speaks with one assistant, even when several separate tasks are running behind it.
How we built it
We designed the entire box in Onshape, laser-cut its acrylic frame, and 3D-printed more than thirty custom parts. Mechanical features include a magnetic handle that lets the camera attach to and detach from the box, a 3D-printed latch, and a carrying handle that doubles as a hard stop for the lid.
A Raspberry Pi runs Clara’s voice interface and touchscreen. For low-level control, it communicates over USB with an ESP32 mounted on a soldered protoboard. The ESP32 reads the lid’s limit switch, controls the three compartment LEDs, and processes data from the MAX30102 pulse sensor. The microphone, GPS receiver, camera, and speaker connect to the Pi through USB hubs; the touchscreen uses HDMI for video and USB for touch and power.
A rechargeable 12 V battery feeds a 12 V-to-5 V buck converter. We split the 5 V output among three USB-C power connections for the Pi and two USB hubs.
Clara’s cloud service runs on Render. Cartesia handles speech transcription and voice output, while Anthropic Claude provides the language and image models. The Pi runs the hands-free conversation loop and serves the touchscreen interface locally. After someone speaks, Core Clara receives the conversation context and the actions currently available. It chooses a short response and a structured action, such as locating a supply, showing a visual, inspecting a requested camera image, or reading the pulse sensor. Photon supports volunteer messaging, and Twilio handles calls to the prototype’s team-owned dispatcher.
How we used our sponsors
CoStar
We built an algorithm to explore where Clara kits should be placed inside an urban building. CoStar data provides the building’s floors, their sizes, and tenant employee counts, which we use to estimate how people are distributed. Matterport scans help us model walking routes on each floor. Our placement agent uses those estimates to recommend kit locations based on travel time.
LangChain
We built Clara’s response coordinator with Deep Agents running on Claude Opus. When an incident starts, a responder scout reviews nearby volunteers and recommends whom to alert. A logistics agent checks supplies in nearby kits and possible rides, while a time-to-care agent compares modeled transport options. The coordinator uses their findings to make decisions through bounded tools while Core Clara continues talking with the person at the box.
Photon
Volunteers get alerts and ride requests by iMessage, WhatsApp, or SMS and reply 1 or 2 to accept or decline, just like in the companion app. Clara also calls from a Photon number. When someone presses SOS, she calls the emergency contact and explains what happened. In the demo, that contact is a teammate playing 911. Afterwards, the same number texts a brief to the contact and to every volunteer on the way. The brief covers injuries, treatment, the time the tourniquet went on, and a link to the live report.
Challenges we ran into
Integrating the hardware was our first challenge. We debugged solder joints, USB power, and pulse sensor processing. Manufacturing was another: we had to complete training and find a makerspace where we could laser-cut our acrylic, all within a few hours.
We also spent substantial time making interaction with Clara feel smooth. Speech, visuals, compartment lights, and background tasks have to arrive in an order that makes sense to the person using the box. We kept the SOS control available without Anthropic so an unavailable agent would not hide it; sending alerts still requires a working connection.
Accomplishments that we’re proud of
- We moved quickly from an Onshape design to a working physical box with more than thirty custom printed parts.
- We connected the Pi, ESP32, lid switch, compartment lights, camera, and pulse sensor into one prototype.
- We finished the hardware early Saturday morning, giving ourselves time to debug the assembled system and build out its software.
What we learned
In a stressful moment, timing matters as much as information. Showing someone how to use gauze before helping them find it makes the guidance harder to follow. We learned to coordinate Clara’s speech, visuals, and compartment lights around what the person needs right now.
What’s next for Clara — Your First Aid Copilot
We want to make Clara more reliable on battery power, expand its language capabilities, and keep more functions available without an internet connection. We also want to learn from medical professionals and emergency responders about how the design should change.
Clara is currently a proof of concept, not a device for patient use. Any real deployment—including pilots in rural areas or refugee settings—would require expert review of its guidance, appropriate hardware validation, privacy protections, and a carefully designed integration with emergency services.
Built With
- 12v-battery
- 3d-printer
- acrylic
- c++
- camera
- cartesia
- claude
- costar
- esp32
- gps
- langchain
- led
- max30102
- microphone
- next.js
- openstreetmap
- photon
- raspberry-pi
- react
- render
- screen
- speaker
- twilio
- typescript

