Inspiration

When the power goes out, most people start counting the hours until it returns. For someone relying on a powered medical device, those hours become a countdown. We were shocked to hear about Judith Greet's case. She’s 71 and uses an oxygen concentrator for her COPD. When Hurricane Beryl hit Texas in 2024, her home lost power. Soon, her device ran out of backup energy, and Judith died. Judith isn't alone. In the US, more than 4.3 million people rely on powered medical devices (HHS emPOWER). But when disasters strike and power is lost, these people face life-threatening conditions. Stories like these make the consequences of a blackout painfully concrete. We wanted to understand what it would take to reach someone like Judith while there was still time to help. So, during HackPrinceton, we called 6 Texas first responders/dispatchers to learn more about this problem. This is what they told us:

"Right now, there's no list of registered patients." "We don't have any pre-emptive calls. Patients right now just call 911 when they need help." "It's hard sometimes, you know. We just have to make decisions on the fly about who to help, and who to prioritize." — League City Dispatch, Texas

Evidently, this is an information-at-scale problem. Because some counties don’t have lists of registered patients, or a pre-emptive plan for these patients for predictable disasters (like hurricanes), no one can track how much backup power each patient has or who hasn't been accounted for yet. This leaves dispatchers being forced to make individual triaging decisions on the fly without context on the entire emergency. Worse, this information is dynamically changing all the time, and worse, underequipped teams can quickly become overwhelmed trying to process this volume of information. That's why we built Hearth: AI that helps responders reach residents before their medical devices lose power.

What it does

When the grid goes down for days, Hearth gives each person on a power-dependent medical device a personalized plan built around their device, building, and condition. It gives first responders a way to run and evaluate thousands of those plans at once. An evacuation has one deadline: when the hazard arrives. A blackout has a different deadline from door to door. The same 6-hour battery that is fine in a shaded brick house for a powered wheelchair user is not fine when it's for a COPD patient's oxygen concentrator on the eighth floor in July with the elevator dead.

  • Three plans, not just one evacuation. Each person gets one of three plans: go through the emergency on their own backup, get supplies at home, or move somewhere with power. The majority of the help that Hearth directs is a crew bringing oxygen cylinders or battery packs to the door. Moving someone is reserved only for when there's no other option.
  • One household, one countdown Hearth estimates how long each household can go without power based on the medical device they use and how much backup power they have left. For households that rely on home dialysis, the countdown tracks when the next treatment is due.
  • Calls before the storm. The county phones everyone on its list in three rounds: a week out, on landfall day, and once the power is out. A registered resident gets a check of their details. Anyone else is asked whether someone at home uses powered medical equipment, and a "yes" registers them on the same call. The device, address and backup hours are each read back, and the address is matched to a CoStar building. The questions about the home come from that building's record and are skipped when the record already answers them.
  • The building layout, down to the door. A crew gets a one-screen brief: stories, walk-up or elevator, where a van can stop, and the floor plan of the unit. With the elevator out, a delivery is budgeted at 10 minutes plus 4 per floor.
  • Dispatch. "I need help", by phone or web, opens a request with no dispatcher click. Claude orders the queue and gives each request a one-line reason. Code then matches all free crews to all waiting requests at once, while considering the patient's mobility: a power wheelchair needs a lift, a bedbound resident a stretcher. The crew has 90 seconds to accept.
  • Relocation that checks for mobility feasibility. A refuge agent searches CoStar for hotels and senior housing within 15 km and outside the dark area. It surveys up to three, and code checks access: a wheelchair or bedbound patient needs a step-free entrance and an elevator above the first floor. Unknown access counts as ineligible. The agent stops for the dispatcher's approval, and a rejection reason is remembered on its next proposal. If nothing passes, a Vexcel aerial search lists shelter leads such as a school gym or a church, as potential places to call.
  • Staging lots. A rule over CoStar's parking fields proposes lots near the outage that are big enough, aren't residents' parking, and sit in a ZIP nobody has asked for help from.
  • Enforced in code, not a model: the model only reads and interprets; code decides and speaks. Nothing tells a resident a crew is coming before a crew has accepted. A model-written acknowledgment is spoken only if it exactly matches one of ten harmless phrases such as "Thank you". When a caller needs help now, the reply leads with 911 and gives no arrival time.
  • Uncertainty stays visible. AI shouldn't be blindly trusted. A finite state machine maps out Hearth's response when prompted with an uncertain request. Every call allows for one clarifying question, and if the uncertainty isn't resolved, then the call is handed off to a human operator. Having no reply is NEVER treated as safe: only the words "I'm safe" record someone as safe, and three silences end the call and escalate.

Implementation & how we integrated sponsors meaningfully

CoStar. CoStar is where every plan starts. When a resident gives their address over the phone, we match it to a CoStar building and skip questions the record already answers. The same floor and elevator details help us estimate how long a crew will spend getting supplies to the door. CoStar decides which questions the registration call asks, which hotels and senior housing count as refuges, how many people a refuge can take (5% of its units, a placeholder), and which lots can stage crews. We read floors, units, year built, construction, elevator and walk-up status, property type, and parking fields from the Houston extract: 1,758 buildings, of which 1,386 are profiled from records alone with no model call. In the replay, removing building data at 20 crews raised the number of patients whose device ran out at home from 184 to 284, and patient-hours at risk from 5,104 to 8,326 (see below on what patient-hours at risk means). This means that CoStar's proprietary data genuinely makes a difference in our routing (hearth/buildings/). That comparison measures the combined building layer. The registration and delivery examples show exactly where CoStar contributes.

Vexcel and Matterport. We use Vexcel to figure out where the van can pull up and which entrance it can approach. We use Matterport to see what the crew will face from there, like entrance steps or a doorway too narrow for their equipment. Both findings go into the crew’s access brief alongside the resident’s own directions, making them critical to Hearth's responses. Vexcel gives roof material and condition, tree cover, the top-down image, parking and entrances. Its vector embedding search quickly finds shelter leads from aerial scenes. Those leads give dispatchers places to call and verify when no known refuge fits. Matterport panoramas give the vision model room tags, door and stair widths, and visible elevators: 1,732 of the 1,758 buildings have a scan. For apartments, the scan represents the unit type. Every profile records how the home's hours-until-unsafe change with each layer removed, so you can see what each dataset changed. This information is critical when determining if a patient can actually relocate given their mobility constraints.

LangChain deepagents. This is the research mastermind in our project. When layers disagree (e.g. CoStar lists an elevator the scan never shows), the coordinator asks the right subagent one follow-up. A dispatcher can see which source supports each finding and which details still need checking before sending a crew. The building agent is a deep agent on Claude. It first drafts a to-do plan and starts three layer subagents (CoStar, aerial, interior) in one turn. Each saves its findings to files that double as the cache. We use LangChain’s FilesystemBackend for those evidence files so later surveys can reuse them. It finishes with one write_profile call. Populated findings require a source from CoStar, Vexcel, or Matterport. The refuge finder is a second deep agent that pauses on interrupt_on for approval, keeps its state in a checkpointer, and reads rejection reasons from a memory file. If a dispatcher rejects a refuge because the entrance has steps, that reason is available to the agent on its next search. On the dispatch page both start on their own: a survey for each new phone registration, and a refuge search for each request code couldn't place. A live survey of a real building takes 55 to 61 seconds. (hearth/buildings/agent.py, hearth/buildings/refuge.py)

Claude. Claude Sonnet 5.5 coordinates the agents, reads images and orders the help queue. Its order counts only if it returns exactly the request ids it was sent; otherwise code's risk-per-hour order is used and the request says so. Claude Haiku 5.5 runs the layer subagents and the call interpreter, which turns each caller turn into structured fields.

Photon & ElevenLabs; Speech & Text A resident can ask for help by phone, get a written record of what they confirmed, and follow that same request as dispatch works on it. Photon's texting services give residents a way to reach Hearth and follow their requests from the phone they already use. Its phone line connects incoming calls and outbound outage check-ins to our ElevenLabs voice agent. After a call, Hearth can send eligible residents an iMessage with what they confirmed, such as how many hours of backup power they have left. If they request help, Hearth sends updates as dispatch progresses and shares crew details and an estimated arrival window only after a crew accepts the job. ElevenLabs carries speech both ways. For its "model" it calls our endpoint, which imitates a chat-model API, so our code chooses every line. Our project utilizes Photon as an extra communication channel to give real-time updates. (hearth/messaging/).

The Calculus of Dispatch. The central planner has two modes, pairing and ALNS. Every mode minimizes the same score: for each person, their risk weight times the hours they wait past the point they are at risk, plus a small term that still makes a not-yet-late need worth a crew. If someone has two hours of backup left and a crew would reach them in three, that’s one hour past their deadline, multiplied by their risk weight. Pairing solves crews against needs exactly (the Hungarian algorithm). We score each eligible crew–request pair by how much waiting cost that crew could prevent per crew-hour spent, then choose the set of assignments with the highest combined score. The closest crew to one household might be the only crew that can serve another, so we make those assignments together. Adaptive large neighborhood search (ALNS) then starts from that pairing and repeatedly removes and reinserts jobs to build multi-stop routes, on a seeded stream so the same inputs give the same routes. It can move a stop to another crew or change the order of stops on the same route. In the county replay, Hearth commits the first stop for each crew and replans as new requests arrive. (hearth/routing.py, hearth/alns.py)

Uncertainty & Finite State Machines. A finite state machine handles uncertainty on every call. The voice model extracts what the resident says into structured fields. Between turns, Hearth’s call controller uses the current state and those fields to decide what to ask next. When a resident reports four hours of backup, Hearth repeats the number and saves it to the household’s record only after they confirm it. If the answer stays unclear or the resident corrects the number, the controller allows one clarification. If it still can’t confirm the duration, Hearth clears the pending number and flags the call for a human operator. This design choice reflects our commitment to human-in-the-loop design that offloads work from operators whenever possible, while simultaneously respecting its limits. Additionally, urgent calls take a separate path. If the resident says they need help now, Hearth escalates immediately and adds a help request for a known resident. Three consecutive silent turns also trigger escalation.

The Beryl replay. Patients per ZIP come from HHS emPOWER (July 2024): 23,273 simulated patients in Harris County. They are 81% oxygen, 9% ventilator, 8% home dialysis and 2% feeding pump. Outages come from EAGLE-I every 15 minutes, and weather is hourly from the Iowa Environmental Mesonet at IAH. The replay runs 11 days in half-hour steps, and a full-county run takes about 6 seconds. Every default sits in one assumptions file with its source and a confirmed flag, and every run copies that file next to its results.

Built with Python, FastAPI, Next.js, React, MapLibre, LangChain deepagents, Claude, ElevenLabs, Photon, CoStar, Vexcel, Matterport, Supabase and Vercel.

Challenges we ran into

Keeping uncertainty in the record. A resident may not know how much backup is left. A building record may lack an elevator field. An unanswered call leaves a household unknown. Each of these had to stay visible to the dispatcher instead of turning into a default that looks like an answer. Hence, a finite state machine that does real work by bringing humans into the loop.

Dirty outage data. The EAGLE-I curve for Beryl had 58 bad intervals: a late onset, a false dip, near-zero readings. We corrected each one and listed every correction.

Two copies of one model. The results page runs the day in the browser, and the Python planner plans the same day. A shared fixture test fails if the two drift apart. It caught a real bug: a step-size change in the browser had silently broken the planner request.

Where it still falls short.

  • The ALNS route search shows no clear gain over pairing in the county sweep, because its plans go stale every half hour as new needs arrive. It does help on the simulation page's single day: weighted waiting fell from 556 to 453 against nearest-first routes.
  • Records-only profiles don't know whether an entrance is step-free, so for a wheelchair or bedbound resident almost no refuge passes. However, we route this request goes to a human operator.
  • No shelter lead has matched a CoStar record in a live run: the Houston kit has no schools or churches.
  • The interpreter over-escalates suspicious callers ("Is this a scam?"), and it took "about 40 hours" at face value.
  • We haven't hand-labeled buildings yet, so the building agent has no accuracy number.
  • 73 of 106 assumptions are still marked unconfirmed.

Accomplishments that we're proud of

On a simulated replay of Hurricane Beryl, with 20 crews, 383 of 647 patients' devices ran out of power at home under today's phone tree. With Hearth's central planner, 11 did.

Crews Today's phone tree Hearth Hearth, central planner
Patients whose device ran out at home (of 647)
5 452 450 436
10 452 437 388
20 383 184 11
Patient-hours at risk
5 104,762 56,104 34,374
10 80,198 26,290 10,822
20 43,841 5,104 208

Each figure is the median of 100 seeded runs, rounded. Patient-hours at risk add up the hours each person spends past the point their device or their room is no longer safe, weighted by risk. A patient counts as ran out once if their backup reached zero in a home without power before a crew moved them. NOTE: Patient-hours at risk add up how long patients spend in modeled unsafe conditions. One patient at risk for two hours contributes two patient-hours. The replay counts time when a patient’s home has lost power and their backup has run out or their home dialysis treatment is due. It also counts time when indoor conditions exceed the modeled safety threshold for that patient. Time spent in a powered refuge is excluded. To calculate risk-weighted patient-hours, the replay multiplies each patient’s time at risk by their modeled risk weight before adding it up.

With 5 or 10 crews, almost every device in the sample still runs out, whichever policy dispatches. Crews are the limit there, and Hearth's gain is in hours: 46% fewer at 5 crews, 67% at 10 and 88% at 20.

"Today's phone tree" is our model of current practice, not the record: staff call the list in order, first come first served, and nobody is moved. "Hearth" ranks the most urgent need first and sends the nearest free crew. "Central planner" uses the same crews and rules, and only changes who goes where. These figures measure modeled power depletion and time at risk, not clinical outcomes. Results · Assumptions

The phone agent, on 58 invented caller turns: backup hours within 10% on 28 of 30, "needs help now" right on 8 of 8, escalation precision 83% and recall 86%, and 20 of 20 guard canaries caught. The median interpreter turn took 1.98 seconds.

On a real phone. Our first rehearsal found two bugs: 13 presses of the storm button placed 13 calls, and a caller who said "Help!" was handed off and hung up on. Both are fixed.

What we learned

  • Deliveries can't fix everything With relocations turned off, Hearth's deliveries technically kept more devices running at 20 crews (272 ran out instead of 383). However, patient-hours at risk were no better than the phone tree's (44,816 against 43,841), because a battery pack can't always replace a dialysis session. Moving people is the largest effect we measured.
  • Who goes where is the second largest. Matching all crews to all needs at once cut patient-hours by 42%, 61% and 93% against nearest-crew dispatch at 5, 10 and 20 crews. The nearest crew for one person may be the only reasonable crew for another.
  • Building data matters most when crews are scarce enough to choose. It cut patient-hours by 8%, 15% and 39% at 5, 10 and 20 crews. Treat that as an upper bound: refuge capacity in the model is a placeholder.
  • A fancier method isn't automatically better. We measured ALNS instead of assuming it helps, and in the sweep it doesn't clearly beat pairing.
  • An allow-list beats a filter. The guard doesn't try to detect bad sentences; it lets ten known-good ones through. It caught all 20 canaries.

FAQ

Is any of this real patient data? No. Patients are generated from public ZIP-level counts (HHS emPOWER). Every record is flagged synthetic, and every view says "simulated".

Does it replace 911? No. It makes the calls a health department doesn't have the staff to make. When a caller needs help now, the reply leads with 911.

How do we know that the model isn't just patting itself on the back? We replay Hurricane Beryl using recorded Harris County outages from EAGLE-I and observed weather from Houston’s IAH station. HHS emPOWER ZIP-level counts guide the synthetic patient population. Hearth and the modeled phone-tree baseline face the same outages, weather, households, and crew count. The replay estimates how many simulated patients would run out of backup power under each dispatch policy. We publish the simulation assumptions alongside the sweep results so readers can see what the comparison depends on.

How does Hearth know whose power is out? The replay assumes per-home outage status from the utility. Most utilities don't share that today, and it is on our list of assumptions.

Why 647 patients and 20 crews? The sweep runs every 36th of the 23,273 patients so the chart reads in tens of crews. We picked that sample size to put Hearth near 5,000 hours at 20 crews, and the choice is recorded in the assumptions file. The same ratio for the whole county would be 720 crews.

Would Hearth have saved Judith Greet or Pamela Jarrett? We can't say that. We can only claim a policy result on simulated patients, with every assumption published.

What's next for Hearth

  • A real deployment's numbers: crews, stock and restock times from a health department and its suppliers, a utility outage feed, and agreements and real capacity at refuges.
  • An accuracy check: hand-label ten buildings and score the building agent against them.
  • A tabletop drill with Texas first responders, then shadow mode: run beside a health department's staff during a real outage before anyone acts on Hearth's plans.
  • Consent and privacy before any real list: agreed with the agency, with access controls and a retention limit.

A medical-needs registry today is a list someone has to phone manually. We want it to become a plan for every person on it, updated as the storm moves.

Built With

  • claude
  • costar
  • deepagents
  • elevenlabs
  • fastapi
  • langchain
  • maplibre
  • matterport
  • next.js
  • photon
  • python
  • react
  • supabase
  • vercel
  • vexcel
Share this project:

Updates

Submission history