Kiosk Landing Page
Presage Scanner at the Kiosk
Nurse's Dashboard
Patient triage check-in
Vultr dashboard

Inspiration

ER wait times are a constant headline, and the scary part isn’t the wait itself, but rather that once a patient is triaged and sent to sit down, nobody rechecks on them until a nurse happens to have a free moment. Vitals can quietly deteriorate, or a stroke can start, and the first sign anyone notices is the patient waving for help. We wanted to build the tool that catches that gap: not another data dashboard, but something that actively watches and speaks up.

What it does

ER Angel is a check-in kiosk that takes a contactless vitals baseline the moment a patient arrives, including: heart rate, breathing rate, EDA (electrodermal activity), and a FAST-inspired facial symmetry check for early stroke screening, using nothing but a camera. Every time that patient rechecks in, the kiosk measures again, and Gemini compares the new reading against that patient’s own baseline, not a generic threshold, to reason about what changed and how urgent it is. If something’s off, a plain-English, prioritized alert lands on the nurse’s dashboard instantly. ElevenLabs gives the kiosk a natural spoken voice throughout, so patients follow a calm voice instead of squinting at on-screen instructions.

How we built it

We split the work by domain: one of us owned the Presage camera-sensing pipeline end-to-end, along with Gemini integration; the other built the backend, dashboard, and voice layer. The architecture ended up as four independent pieces talking over HTTP: a local capture agent that owns the physical camera (Presage’s SDK needs direct OS-level camera access, so it can’t run in a browser or on a cloud server), a backend that stores readings and calls Gemini for risk scoring, a kiosk frontend for patients, and a dashboard for nurses.

Stack: React + Vite (kiosk and dashboard), Node.js + Express, SQLite, Docker Compose, Caddy, Terraform.

Gemini, used twice per rescan. ** The first call returns structured JSON (risk level, what changed, recommended urgency). The second turns that into a one-line alert a busy nurse can scan in two seconds. Both prompts forbid diagnosing or recommending specific tests or treatments. If Gemini is slow or out of quota, calls retry with backoff, then try a backup model and a second key, and finally fall back to transparent rule-based alerts, so the dashboard never shows a blank or broken card.

ElevenLabs, built for reliability. ** Every spoken line is generated once and cached in memory, on disk, and in cloud object storage. If ElevenLabs is unreachable, the kiosk falls back to the browser’s built-in speech, so it’s never silent.

Vultr, as real infrastructure. Everything is defined in Terraform: a Vultr Cloud Compute instance in Toronto (so the live patient database stays in Canada), a reserved IP, a firewall exposing only SSH/HTTP/HTTPS, and Vultr Object Storage holding the voice cache and nightly database backups, with a restore we actually tested. Caddy provides automatic HTTPS for two separate apps: a public kiosk and a login-protected nurse dashboard. A one-command deploy script (plus GitHub Actions) pulls, rebuilds, and health-checks the server.

Security, because it’s health data on the public internet. ** ** The nurse dashboard requires a login. The public kiosk can only reach an allowlist of endpoints, can’t read patient names, and is rate-limited per IP. The voice endpoint only speaks the kiosk’s own lines, Gemini calls are capped per hour, and request logs are redacted so no secret ever lands on disk.

Challenges we ran into

The Presage SDK’s actual data shapes weren’t documented anywhere we could find, so almost every signal, heart rate, breathing rate, facial landmarks, even the validation codes explaining why a reading failed, had to be reverse-engineered live: request a metric, dump the raw decoded object, and read the real field paths off our own console logs before writing a single line of extraction logic. We got burned more than once trusting an assumption instead of testing it; at one point, we treated every SDK validation message as a “framing problem,” which included a totally normal “hold still” reassurance message, and it silently froze our capture countdown at 30 seconds for a while before we caught it. Getting the browser and the local capture agent to cooperate over the same physical webcam without fighting each other for camera access was its own separate puzzle.

Accomplishments that we’re proud of

Getting real, live stroke-screening data out of a consumer webcam is the thing we’re most proud of — not just heart rate, which plenty of hackathon projects do, but a genuine multi-signal fusion (vitals + facial asymmetry + EDA) feeding into one AI-driven risk judgment, exactly the kind of thing the Presage challenge is looking for. We’re also proud of the adaptive capture logic we ended up building: instead of a dumb fixed-length recording, the capture agent now only counts time toward its target while framing is actually good, so a patient who’s briefly out of frame still ends up with a full, usable reading instead of 30 seconds of garbage data.

What we learned

Don’t trust an SDK’s documentation gaps to mean “figure it out later” — build the diagnostic tooling first, even before the feature you actually want. Every time we skipped that step and guessed at a field name or a validation code’s meaning, it cost us more time than the diagnostic logging would have. We also learned a lot about how much design work goes into making an AI health signal trustworthy rather than just technically working — comparing against a patient’s own baseline instead of an absolute threshold, and being explicit in both the UI and the Gemini prompt that facial asymmetry is a screening cue and not a diagnosis, turned out to matter as much as the detection code itself.

What’s next for ER Angel

Clinical validation is the obvious next step; our asymmetry and EDA thresholds are calibrated from a handful of our own test captures, not a real dataset, and we’d need to close that gap before this touched a real patient. Then, we’d like to see our software in real waiting rooms!