OAH Data Doctor

Inspiration

We opened the anchor record of the OneAquaHealth FHIR sandbox: the 2013 water temperature of the Almyros stream in Crete. It reports an annual mean of 198,000 °C next to a median of 19.8 °C, and the server's own FHIR validator answers "No issues detected during validation."

FHIR conformance checks shape (structure, cardinality, types, codes). It cannot notice that a median must lie between its own minimum and maximum, or that liquid water cannot be 198,000 °C. On the sandbox, 125 of 385 official records cannot be right as published, and all 125 pass the server's own $validate. One Health research joins water, air and health data across cities, so one silent scale error can become a published trend, a wrong comparison or a false causal claim.

FHIR tells you whether data is shaped correctly. Data Doctor tells you whether it makes scientific sense, and what you can safely conclude from it.

What it does

OAH Data Doctor is a read-only evidence-integrity layer above FHIR, built for Track 7 – Digital Health Standards. It does not replace FHIR validation; it adds what conformance cannot check, on top of OneAquaHealth's own FHIR Implementation Guide and sandbox:

1. Doctor: can this value be true? 20 deterministic rules check statistics, physics and definitions, and point to the exact field (FHIRPath), the published values and the constraint they break, for example \( \min \le \text{median} \le \max \). Every finding writes its proof out:

$$\frac{\text{average}}{\text{median}} = \frac{198{,}000}{19.8} = 10{,}000 = 10^{4}$$

The rule allows a factor of at most 100; here it does not hold. Root cause: unknown, and nothing is ever corrected.

1. Trace: what would it contaminate? The data sets, annual series, comparisons and claims that use a flagged record. It never estimates.
2. Compare: can two numbers be compared? Eight dimensions (measure, unit, medium, population, period, aggregation, method, integrity) give direct, conditional, not comparable or blocked until reviewed.
3. Claim guardrail: can I say this? A researcher's sentence is judged (supported, conditional, unsupported, blocked), shown on an evidence ladder (observation → description → comparison → association → causation), and given wording the data does support.

Every finding carries provenance (server, retrieval time, rule version, the record's sha256) and, where known, the line in OneAquaHealth's own source spreadsheet where the values first appear. Results export as FHIR OperationOutcome (validated against R4), JSON and a report that separates what was observed, derived, inferred and unknown. The core audit is deterministic; AI is an optional explanation layer that never decides a finding or a verdict.

Who it is for: data stewards publishing OneAquaHealth data (a review list with the exact field and source line); One Health researchers (which records, comparisons and claims hold up); FHIR integrators (oah_audit.py check exits with an error status, so it can gate a pipeline); monitoring and citizen-science teams (check a file in the browser before sharing it).

Impact on ecosystem and human health, measured on the sandbox:

• Water ecosystems: 121 of 141 official stream-water records from Crete are not safe to use as published, including dissolved oxygen, nitrate, nitrite, temperature, pH and dissolved metals (mercury, lead, cadmium, arsenic): the conditions aquatic life depends on.
• Air: 4 of 103 Benevento air-quality records report PM2.5 above PM10, which cannot happen.
• Human health: the 141 health records have no errors; the risk is in how they are compared and what is claimed (age bands that do not overlap, causal claims aggregated data cannot support), which Compare and the claim guardrail address.

How we built it

• Backend: Python 3.11, FastAPI and Pydantic, with a read-only async FHIR client (GET and $validate only, paging, retries, cache) and sha256-verified offline snapshots.
• Knowledge: derived reproducibly from the OneAquaHealth IG source at a pinned commit.
• Rules: pure, versioned functions with precision-aware tolerances. Every value is compared within half a unit of its last published decimal, so rounding alone never raises a finding.
• Statistics: exact small-sample methods (Mann–Kendall permutation test, Theil–Sen slope) for trends and comparisons.
• Console: React + TypeScript (Vite, Tailwind); every finding opens with its evidence chain.
• Optional language model (local Ollama, any OpenAI-compatible endpoint, or Claude): it may only rephrase a computed finding. Any number, cause or correction it adds is detected, and the text is discarded.

Challenges we ran into

• The sandbox vanished. Its DNS record disappeared for about a week during the hackathon, so we built verified offline snapshots with automatic fallback; every screen says whether you see live or snapshot data.
• No profiles to validate against. The IG website was offline and the server hosts no profiles, so we derived the checkable constraints from the IG's FSH source.
• Rounding. Naive checks raise false alarms on rounded values; property-based tests prove that no real sample, rounded at 0–4 decimals, triggers an identity rule.
• Our own false positives. Reviewing live findings caught two rule defects, and a fault-injection test caught a third. All were fixed with regression tests.

Accomplishments that we're proud of

• 346 findings (125 critical) on real OneAquaHealth data. The same values appear in the IG's own source spreadsheet, so the problems predate FHIR conversion.
• 271 of 271 planted faults detected, and 0 new findings on 100 records after harmless transformations. That measures rule sensitivity, not real-world accuracy, and we say so.
• 267 backend tests and 38 end-to-end tests (offline, WCAG 2.1 AA accessibility in light and dark), green in CI.
• Honesty by construction: no hand-typed numbers (the docs are rendered from an audit run), root cause always "unknown", read-only access, and live and snapshot data always labelled.

What we learned

A validator answers "is this well-formed?"; researchers need "can this be true, and what can I say?". Small, provable checks (order statistics, subset relations, partitions) find more real problems than large generic rule sets. Showing where a value first appears is far more useful to a data owner than guessing why it is wrong.

What's next for OAH Data Doctor

• Run Data Doctor as a $validate-style FHIR operation, so any OneAquaHealth server returns scientific findings next to conformance issues.
• Add a check step to the IG's "Publish & Validate" workflow for new cities, stopping datasets that are well-formed but scientifically impossible.
• Publish uncertainty (sample size, confidence intervals, coverage) in OAH Observations, so differences between cohorts can be tested.
• Work with the data owners on the Almyros chemistry summaries and the Benevento site coordinates.

Data: OneAquaHealth FHIR sandbox (HL7 Europe), included with the organisers' permission; OneAquaHealth is funded by the EU's Horizon Europe programme. Independent project, not endorsed by OneAquaHealth, HL7 Europe or IEEE. Built with help from an AI coding assistant (Claude); the product uses no AI to decide findings. The video narration is synthetic speech (ElevenLabs).

Built With

• axe-core
• fastapi
• fhir-r4
• fhirpath
• github-actions
• hapi-fhir
• hl7-fhir
• httpx
• hypothesis
• ollama
• playwright
• pydantic
• pytest
• python
• react
• sqlite
• tailwindcss
• typescript
• vite