-
-
SentinelRx combines local AI note interpretation with deterministic medication-record checks. AI proposals never hide discrepancies.
Inspiration
A medication missing from a discharge list could represent an error, an intentional stop, or incomplete information. A confident AI interpretation cannot settle that distinction on its own.
SentinelRx brings the medication records and the accompanying note into one review workflow, keeping source evidence and uncertainty visible.
What it does
SentinelRx flags potential omissions, additions, duplicate records, and dose-text changes in synthetic FHIR R4 medication data.
A deterministic gate checks source references, medication identity, transition roles, and bundle-local discrepancy predicates. Findings appear as EVIDENCE-CHECKED or ABSTAIN. Evidence-checked does not mean clinically correct.
A local machine-learning model separately proposes whether a short medication note describes stopping, continuing, changing, or starting treatment. It abstains when model separation is insufficient or the wording triggers uncertainty guards.
The reviewer manually links the note to a medication and sees the exact source text alongside the record finding. An AI proposal cannot hide, resolve, or upgrade a discrepancy.
The demo also includes:
- A medication-transition timeline and source-evidence view.
- Reviewer annotations and a downloadable audit containing source hashes and model version.
- A Counterfactual Lab showing how findings change when the underlying synthetic records change.
How we built it
The application uses Python, Streamlit, FastAPI, and Pydantic.
The note classifier uses scikit-learn character TF-IDF features and logistic regression, trained on 48 published synthetic examples. Inference runs locally without an external model API.
The record-analysis path uses namespace-safe medication identities based on terminology system and code, explicit patient references, supported-order checks, and deterministic discrepancy predicates.
The two paths remain separate: the learned model interprets text, while record findings depend on structured evidence.
Challenges we faced
Medication identity cannot safely rely on a code without its terminology system. We therefore preserve both in the canonical identity.
Another challenge was deciding how much a verification label could honestly promise. We use EVIDENCE-CHECKED to describe checks on supplied records without implying clinical validation.
Missing medication identity can invalidate an absence claim, while a supported positive discrepancy for another identified medication may still be assessable. We refined this distinction rather than treating every incomplete record identically.
What we learned
AI abstention and evidence checking answer different questions. A model may propose an interpretation of a note, but that does not establish that the note belongs to the right medication or that the transition is clinically appropriate.
We also learned to separate software regression results from evidence of real-world usefulness. Perfect agreement on generated rule-based examples is not clinical accuracy.
What we tested
78 automated tests pass, covering record checks, candidate tampering, input handling, API limits, and isolation between AI proposals and record findings.
On a small author-written synthetic evaluation split excluded from model fitting:
- 23/24 outputs matched the expected labels, including abstentions.
- The system made proposals on 15/24 notes.
- All 15 proposals matched their authored labels.
- One positive example was abstained on.
- All eight examples labelled for abstention were rejected.
These sentences were authored during the same development process as the training data. They are not independently labelled or clinically validated. The repository includes individual predictions and data hashes.
The record path also matches expected outputs on 120 generated regression cases, 90 generated edge/incomplete cases, and 32 authored perturbations. These suites repeat known patterns and measure software behavior, not clinical generalization.
Prior-work disclosure
My earlier project, FHIRGuard, explores the same medication-reconciliation, provenance, abstention, and constrained-model concepts. Those concepts are prior work and are not claimed as new inventions.
SentinelRx is a separate compact implementation focused on interactive review. This revision adds a local learned note classifier, a manually linked note-and-record workspace, source-hashed review exports, and stricter record checks. Repository history documents the implementation dates.
Limitations and next steps
SentinelRx is a research prototype using synthetic data only.
Transition sides use explicit project tags; encounter chronology and list completeness are not independently established. Dose comparison is textual rather than semantic. The detector and evidence gate share parser assumptions.
The small note classifier can make mistakes, and its scores are not calibrated clinical confidence. Reviewer annotations are demo records, not authenticated clinical sign-offs.
Next steps are independent clinician-authored evaluation, richer encounter and medication-list semantics, and usability testing with intended reviewers.
AI reads the note. Rules check the records. The reviewer decides.
Log in or sign up for Devpost to join the conversation.