Inspiration
Utah DOT publishes a work zone feed under the federal WZDx specification. Run it through USDOT's own validator and it passes with zero errors.
The same feed asserts 744 work zones currently active, and every one of them has an end
date that is already in the past. Its update_date has not moved in over three years.
A navigation service consuming that feed gets 744 phantom closures, and every automated check in the federal pipeline says the data is fine. Conformance and trustworthiness are different properties, and only one of them was being measured. That gap is the project.
What makes it a governance problem rather than a data-cleaning one is who publishes it. Forty independent organizations, forty-one feeds between them, each a sovereign public agency. You cannot fix their data, you cannot make them fix it, and you cannot opt out of consuming it. The only decision available is whether to believe each one today, and that decision has to be made continuously, defensibly, and the same way every time.
What it does
Interchange produces two things. For the data consumer, a merged work zone feed that carries only data from publishers currently earning trust, validated against the official schema before every publish. For the registry owner, an evidence packet: what a publisher got wrong, the observations proving it, and a drafted notice a human can approve and send.
Underneath, one agent per publisher across the 41 feeds in the federal registry, each
polled on the cadence it declares for itself. The trust decision is deterministic. Six versioned rules score reachability, freshness,
conformance, internal contradiction, and churn, and the verdict is the highest severity
currently firing: ADMIT, WATCH, or QUARANTINE. Escalation is immediate. Recovery
requires a hysteresis window, so a publisher cannot flap back into admission on one good
poll.
Zones that several publishers describe are collapsed into one, with every contributing source retained. Matching is by how much of each zone the other covers, not by how close they are; the negative controls under Accomplishments show why that distinction earns its keep.
Free text from publishers is treated as hostile input and screened before it can reach a model, a notice, or the republished feed. When the screener is unavailable the text is redacted and flagged exactly as if it had been blocked, so an outage on our side cannot quietly become a pass.
Findings become evidence packets: what is asserted, the observations behind it, and a
drafted notice. A packet sits in DRAFT until a human with the approver role moves it to
APPROVED or WITHHELD. No notice leaves the system on a model's say-so.
The merged feed is validated against the official WZDx 4.2 schema before it is emitted, and withheld if it fails. A merged feed that would quarantine its own publisher is the one failure this project cannot ship.
How we built it
Google Cloud throughout. Firestore holds the fleet's durable state: publisher records, retained observations, the canonical source map, evidence packets and daily rollups. The live runner is a Compute Engine VM under systemd, polling on a 900 second cadence. The operator console is Next.js on Cloud Run behind Google sign-in, with viewer and approver roles. The merged feed is written to Cloud Storage. Model Armor screens publisher text. Gemini runs through Vertex AI with the VM's own service account, so no API key exists in the deployment to leak.
The architectural decision worth defending is where the model is not. Gemini appears in exactly two places: adjudicating duplicate pairs that the deterministic matcher cannot resolve, and drafting notice prose that a human then approves. It has no path to a trust verdict. We do not request a confidence score from it either, because a scalar invites a threshold and a threshold puts the model back in the gate.
The second decision is that absence is never recorded as a pass. A rule that cannot be
evaluated returns NOT_APPLICABLE and records why. An unresolvable schema version records
SCHEMA_UNKNOWN and suppresses the affected rule instead of failing the publisher. A
key-gated publisher is NO_ACCESS and is excluded from coverage denominators rather than
counted as passing. Every cap and truncation is stated in the output.
Challenges we ran into
The honest one is that a system meant to run for weeks kept finding new ways to stop.
The canonical source map is append-only, and it grew to 303,962 documents. Reading it whole
at startup was a single unbounded Firestore query, and once the collection crossed some size
the query began timing out with a 503. The client library's retry path then raised an
AttributeError instead of retrying, so a recoverable timeout became a fatal crash inside
startup, before the first poll. The runner crash-looped 2,468 times over two days. Reading
the collection in 2,000 document pages returns all 303,962 in 81 seconds where the single
query fails after 64.
The VM has also run out of memory twice. The first time was the single cycle that rewrote all 49,000 zones after a screening change: Python does not return freed memory to the operating system, so the next cycle started from that high water mark, and we read it as a one-off transition cost rather than a leak. A second OOM on 2026-08-23 says that reading was too generous, and the machine size is now an open question rather than a settled one.
Both only appeared because the thing had been running for weeks. Neither would have shown up in a demo, and neither was in any test.
Accomplishments we are proud of
The negative controls are what we would point at first. A matcher that accepts everything looks impressive on the flagship pair and is worthless, so two controls run as tests against real data. Missouri DOT against St. Charles County: four candidate pairs fall inside the distance threshold and three of them intersect at zero metres, yet all four are different work zones; one is a 4.8 km ramp closure lying inside a 33 km pavement corridor. Coverage scores them 0.039 to 0.075 against a 0.6 threshold and rejects all four, where proximity alone would have merged them. And CivicLink against Missouri DOT: overlapping bounding boxes, zero candidate pairs.
The whole pipeline also reproduces offline, against a checksummed snapshot of the live feeds, in three commands and with no cloud account. All 473 tests run that way.
We also got strict about numbers. Every quantity in the README has a committed command that prints it. Three review rounds caught figures that had been measured in throwaway scripts and written up as though the committed probes produced them, so the rule became: commit the code that measures it, or do not write the number down.
What we learned
Live data moves more than you plan for. NJIT's coverage ranged from 87.3 to 100.0 percent across five runs in two days. New York DOT carried 6,848 features when the research was written and 6,299 when the reconciler was re-measured. No test in this repository asserts an exact live count. They assert shape and direction, and the figures are re-measured at the point of use.
What's next
Thirteen of the registry's feeds are key-gated and have never been probed. They show as
NO_ACCESS in the console rather than being assumed fine, their data quality is unknown,
and every coverage claim is scoped to the openly reachable publishers.
Try it
The console is live at https://interchange.sawinyh.com. Sign in with any Google account; every account gets read-only viewer access, enforced by Firestore security rules that deny all client writes. Start at the fleet board, open Utah DOT, then the notice queue, then Output health. The approve action is restricted to an allowlisted approver because approvals write to the live audit trail, so the demo video shows that step instead.
Scheduled polling is suspended during judging and the console's masthead says so, with the date the data runs through. The history on screen is the real thing: hundreds of cycles against 41 live feeds, collected before the pause.
The full write-up is on the builder's blog: https://sawinyh.com/blog/conformance-is-not-trustworthiness/
Log in or sign up for Devpost to join the conversation.