TriageX — One Surge. Two Futures.
What if hospitals could respond to an emergency surge before their emergency departments became overwhelmed?
A major accident doesn't just create more patients. It creates a chain reaction: emergency departments fill up, ambulances wait longer to hand off patients, and staff scramble to coordinate additional capacity.
The problem isn't always a lack of information. Hospitals already have dashboards and alerts. The missing piece is often what happens after an alert.
We built TriageX to close that gap.
TriageX is an AI-powered emergency department surge forecasting and staffing coordination prototype that predicts operational pressure, recommends an intervention, and uses a real FaceTime AI agent to help coordinate the response.
Instead of stopping at "a surge is coming," TriageX helps answer "what can we do about it?"
Inspiration
We wanted to explore a simple question: What if AI could move beyond predicting a problem and actually help coordinate the people needed to solve it?
Emergency departments operate under intense time pressure. A single mass-casualty event can affect multiple hospitals, and even a short delay in arranging additional staffing can contribute to overcrowding and ambulance handoff delays.
We saw an opportunity to combine predictive simulation, human oversight, real-time AI conversation, and live video into one connected workflow.
Our goal wasn't to replace hospital staff or automate clinical decisions. It was to demonstrate how AI could support operational coordination while keeping humans in control.
What It Does
TriageX simulates five hospitals in metro Atlanta responding to a major accident on I-85.
The system tracks emergency department conditions, forecasts congestion, and identifies when additional staffing could reduce pressure.
Here's the workflow:
- Detect the surge. A simulated accident increases emergency department demand across the hospital network.
- Predict the impact. TriageX forecasts how congestion and ambulance handoff delays may develop.
- Recommend an intervention. The system proposes additional staffing at the affected hospital.
- Keep a human in control. An operator reviews and approves the staffing response.
- Coordinate through FaceTime. A real caller connects to our AI agent through Chert's FaceTime integration. Gemini powers the conversation and interprets a fictional staff card shown through the camera.
- Validate and confirm. Our backend checks the extracted card information against a synthetic staff roster and requires explicit verbal confirmation.
- Compare two futures. TriageX simulates the hospital's outcome with and without the staffing intervention.
The result is a workflow connecting prediction → approval → real-time communication → verified response → measurable simulated impact.
The Results
In our deterministic simulation, confirming three additional staff members produces:
- 36% fewer simulated ambulance handoff hours
- 9.25 ambulance handoff hours avoided
- 25.50 hours in the baseline scenario versus 16.25 hours with intervention
- 45 minutes faster simulated recovery
Our demo includes a genuine live FaceTime interaction, while the remaining two staffing replies are explicitly simulated to demonstrate the complete three-person intervention.
How We Built It
Chert + LiveKit — Real FaceTime Interaction
Chert connects an actual FaceTime call to our AI agent through LiveKit.
Instead of presenting a prerecorded chatbot interaction, we demonstrated a real caller communicating with the agent and presenting a fictional staff card through the phone camera.
This creates a natural interaction using technology people already know how to use.
Gemini — Voice, Vision, and Operational Intelligence
We integrated Gemini to power live conversational interaction and analyze video frames from the FaceTime call.
Gemini extracts card details, while our FastAPI backend independently validates the information against the synthetic roster. The agent also requires explicit spoken confirmation before a staffing response is recorded.
We additionally use Gemini to generate grounded operational summaries.
Tiger Data — Time-Series Operational History
We use Tiger Data with PostgreSQL and TimescaleDB to persist simulated hospital metrics, staffing responses, intervention history, and baseline-versus-intervention comparisons.
This allows us to examine how hospital conditions change over time rather than relying only on a single dashboard snapshot.
Entire — AI-Assisted Development Provenance
We used Entire to capture development checkpoints and connect AI-assisted engineering sessions to Git history.
This helped make our rapid development process more traceable and reproducible.
Application Architecture
Our system combines:
- React + TypeScript for the real-time operational dashboard.
- FastAPI + Python for simulation control, staffing workflows, and validation.
- WebSockets for live dashboard updates.
- Gemini for conversational and visual AI.
- Chert + LiveKit for real FaceTime connectivity.
- Tiger Data + TimescaleDB for persistent time-series data.
- Entire + Git for development provenance.
The simulation uses a fixed seed to make scenarios reproducible, allowing us to compare interventions against the same baseline conditions.
Challenges We Faced
Making live FaceTime AI actually work. Integrating a real call with an AI agent required coordinating video, audio, conversation state, and backend actions. We had to ensure the agent could respond to what the caller showed it, not just what the caller said.
Turning AI interpretation into a trustworthy action. Reading a staff card is different from authorizing a staffing response. We separated Gemini's interpretation from backend roster validation and required explicit verbal confirmation before changing the operational state.
Connecting the real world to a simulation. One of our biggest challenges was ensuring a staffing confirmation could influence the simulation in a meaningful and reproducible way, with appropriate lead time before staff became available.
Building a coherent end-to-end demo. Forecasting, approval, FaceTime, validation, persistence, and the dashboard all needed to work together. Testing individual components was only part of the challenge; the real difficulty was connecting them into one understandable experience.
What We Learned
We learned that building an effective AI agent isn't just about making the conversation sound intelligent.
The important engineering decisions happen around the model: what information it can access, what it is allowed to change, what must be independently verified, and when a human needs to approve an action.
We also learned how powerful real-time video can be as an interface. A camera can become a practical input for an operational workflow, not just a communication channel.
Most importantly, we learned that the value of a prediction depends on whether someone can act on it.
What's Next
TriageX is currently a proof of concept built around synthetic hospital data and fictional staff identities.
Our next steps would be to expand the staffing coordination workflow, improve multi-hospital resource allocation, add more robust identity and authorization controls, and evaluate the system with realistic operational constraints and appropriate domain expertise.
Longer term, we'd like to explore how this approach could support other time-sensitive coordination problems, including disaster response and critical infrastructure operations.
A surge alert doesn't fill a shift. TriageX connects the prediction to the response.
One Surge. Two Futures.
Built With
- ai-agents
- chert
- entire
- fastapi
- gemini
- gemini-live
- git
- github
- javascript
- livekit
- node.js
- playwright
- postgresql
- pydantic
- python
- react
- recharts
- tailwind-css
- tiger-data
- timescaledb
- typescript
- vision
- vite
- websockets