SignalGap

Inspiration

Emergency department waiting rooms are constantly changing. A patient may be assessed when they arrive, but the information used for that decision becomes older as they continue to wait.

A patient can appear stable simply because their breathing rate, oxygen level, or heart rate has not been checked again. A normal risk score can show who appears most concerning based on the information currently available. It does not show where collecting fresh information may change the next decision.

I built SignalGap to explore that problem.

SignalGap helps identify the waiting room patient who may look stable only because no one has looked again.

What it does

SignalGap is a research prototype that helps an emergency department team decide which waiting patient should receive a standard reassessment next.

It separates two questions.

The first question is how risky the patient appears based on the information currently available.

The second question is how valuable it would be to collect fresh information from that patient.

These questions are related, but they are not the same. A patient can have a moderate current risk estimate while still being the most useful person to reassess because their observations are old, missing, or likely to change the decision once updated.

The live board contains a fictional shift of 14 patients. SignalGap shows each patient's current estimated risk, clinical risk rank, observation freshness, missing information, uncertainty, reassessment value, recommendation, and audit history.

The main demonstration focuses on Maya, a fictional 67 year old patient waiting with shortness of breath.

Before reassessment, Maya has an estimated risk of 4.1 percent and a clinical risk rank of sixth. A system that only sorts patients by current risk would not place her first.

However, Maya's respiratory observations are 87 minutes old. SignalGap estimates that collecting fresh observations could significantly change her risk estimate or position relative to other patients. This places her first in the reassessment queue.

The user begins a respiratory reassessment and enters an oxygen saturation of 86 percent, a respiratory rate of 28 breaths per minute, and a heart rate of 108 beats per minute.

SignalGap validates and stores those observations, rebuilds the patient features, reruns the model, recalculates uncertainty, and updates both rankings.

After reassessment, Maya's estimated risk rises from 4.1 percent to 13.4 percent. Her clinical risk rank rises from sixth to second. At the same time, her reassessment queue position falls from first to tenth.

The rankings move in opposite directions for a clear reason.

Before reassessment, Maya ranked first because collecting fresh information was valuable. After reassessment, that information gap has been resolved. Her current risk is now clearer and higher, so the next action changes from gathering information to requesting clinician review.

SignalGap does not make the final clinical decision. The user records the action, and the new observations, model update, recommendation, and human decision remain visible in the audit history.

How we built it

SignalGap includes a complete backend, frontend, modeling pipeline, database, evaluation workflow, and automated test suite.

The backend is built with FastAPI and Python. It handles patient state, input validation, feature calculation, model inference, conditional simulation, queue ranking, audit events, human decisions, and failure behavior.

SQLite stores the local scenario state and audit history. This makes the demonstration reproducible and easy to reset.

The frontend is built with Next.js, React, and TypeScript. It includes the live patient board, patient details, reassessment form, before and after comparison, evaluation page, model card, and audit timeline.

The modeling pipeline creates a reproducible synthetic dataset of 5,200 encounters. The data includes age, acuity, complaint type, arrival mode, workload, physiology, observation age, and patterns of missing information.

The data is divided into separate training, calibration, and test groups. The main model uses histogram gradient boosting with probability calibration. Eight bootstrap models provide a prototype indication of uncertainty.

When a patient has stale or missing observations, SignalGap generates plausible fresh values using only the information available at that time. It repeats this process 80 times and measures how often new information could change the review threshold, estimated risk, or patient ranking.

Evaluation labels are stored separately from operational patient records. The live ranking system cannot access those labels. Regression tests confirm that changing an evaluation label or patient identifier does not change the model output, simulation, recommendation, or queue position.

Challenges

The biggest challenge was separating current risk from the value of collecting new information.

At first, these ideas can seem contradictory. A patient can move higher in the clinical risk ranking while moving lower in the reassessment ranking. The interface needed to explain this clearly without making the workflow confusing.

The before and after comparison became the most important part of the application because it makes the distinction visible. Maya moves from moderate estimated risk to higher estimated risk, while her reassessment priority drops because the missing information has already been collected.

Another challenge was preventing evaluation leakage. A benchmark is not meaningful if hidden answers influence the system being tested. I isolated the evaluation labels from all operational logic and added tests proving that they cannot affect the live system.

Communicating uncertainty was also difficult. The bootstrap range is useful for showing model instability and the effect of missing information, but it is not a clinically validated confidence interval. SignalGap describes it as a prototype uncertainty indicator and presents that limitation clearly.

Failure behavior was another important part of the project. SignalGap includes a fictional case where model inference fails. Instead of showing an old prediction, the system marks the prediction as unavailable, preserves observation freshness, uses a documented fallback order, and directs the user toward standard protocol.

Evaluation

The evaluation page reads from a benchmark file generated by the project code.

In the frozen synthetic workflow scenario, SignalGap captures both hidden evaluation cases within the top three reassessment positions.

SignalGap reaches a hidden case after a median of 2.5 reassessments. The ESI plus wait comparison requires 3 reassessments, while risk only ranking requires 4.

SignalGap has a top three capture rate of 100 percent in this synthetic scenario. The ESI plus wait and risk only comparisons each have a top three capture rate of 50 percent.

The calibrated model has an expected calibration error of approximately 0.017 on the synthetic test set.

These results describe the behavior of the prototype on generated data. They are not evidence of clinical effectiveness, patient benefit, or real world performance.

What we learned

Building SignalGap showed me that missing information can be important without automatically meaning that a patient is in danger.

It also showed me that risk prediction and information collection should be treated as separate decisions. A patient may not currently rank as highest risk, but collecting one fresh observation may still have a large effect on what happens next.

I also learned how important it is to isolate evaluation data from operational logic, communicate uncertainty honestly, design visible failure behavior, and keep final clinical decisions in human hands.

What is next

SignalGap is a research prototype and is not intended for clinical use.

Future work would require clinician designed reassessment policies, external data, prospective workflow testing, subgroup analysis, human factors evaluation, security controls, and integration with governed clinical systems.

The central idea is simple.

Do not only ask who appears highest risk. Ask where one more observation could change what happens next.