zipsick

Catches ZIP-level illness outbreaks early by watching open-web signals before people reach the hospital.

Comment

Inspiration

People get sick before they go to the hospital. They post on Reddit first. They file a 311 complaint. They leave a one-star Yelp review about the chicken. By the time an Emergency Department cluster shows up in official surveillance, the community has been signaling for days.

Official systems like CDC NSSP/BioSense are solid — they're just downstream. Zipsick watches the step before.

What It Does

Zipsick is an autonomous agent that runs the full loop without a human in it:

  1. Ingest — Nimble fetches public web pages. NYC Open Data delivers 311 complaints. Reddit and Yelp add community signal.
  2. Extract — A regex parser pulls NYC ZIP codes and symptom keywords from every piece of text.
  3. Store — Every signal lands in ClickHouse with a run_id, a ZIP, a symptom type, and a synthetic flag that's honest about what's real and what's a controlled test.
  4. Score — The anomaly engine runs a Z-score query against a 90-day rolling baseline:

$$z = \frac{x_{\text{recent}} - \mu_{\text{baseline}}}{\sigma_{\text{baseline}}}$$

A Z-score above 2.5 with at least 5 signals in the 6-hour window triggers a candidate alert.

  1. Verify — Before calling anything an outbreak, the clinical verifier calls a FHIR endpoint with _summary=count. One aggregate integer comes back. No patient IDs, no names, no clinical notes. Just a count.
  2. Act — Confirmed alerts fire a Slack message and a Datadog log tagged with alert_id and run_id. Every stage is traceable end to end.
  3. Publish — Senso publishes the confirmed alert as a citable markdown document at cited.md. If Senso credentials aren't set, it falls back to a local public/alerts/{id}.md file and logs which path it took.
  4. Monetize — GET /alerts/confirmed/{id} returns 402 Payment Required with x402 payment instructions. Add x-payment: demo-paid and you get 200 with the full alert package and payment_status: paid.

How We Built It

The backend is FastAPI. ClickHouse runs the anomaly SQL — the Z-score query joins a 6-hour recent window against a 90-day baseline in a single CTE. Nimble handles the open-web fetches with IP rotation. The FHIR adapter hits a real endpoint in production and falls back to a hardcoded aggregate lookup during testing so unit tests don't make live network calls.

The frontend is a Next.js dashboard that proxies /api/* to the FastAPI backend. It polls /status every 4 seconds and shows the proof checklist, latest alerts, a ZIP-level heatmap, and a live x402 payment flow.

Everything is tagged with a run_id so a single Datadog filter shows the full pipeline from ingestion through payment in one trace.

Challenges

Getting the synthetic flag right was harder than it sounds. In early runs, Reddit mock posts were being stored as synthetic=false because the mock flag wasn't propagated all the way into extract_signals(). The anomaly engine would pick them up and Z-scores would look artificially clean. Fixed by threading the synthetic parameter through the whole ingestion stack and making the NYC 311 lane — the only lane that's always real data — hardcode False explicitly.

The Z-score was hitting 38x–98x during testing because accumulated spike injections piled up in the 6-hour anomaly window. Built a reset script that clears the window before each run so the math stays honest.

The Nimble request body had "parse": True instead of "render": True, which caused 500 errors against the live API. Small thing, took too long to find.

The x402 /status endpoint had a shallow copy bug — dict(RUN_STATE) doesn't copy nested dicts, so mutations inside the route were leaking back into the global state. Fixed with copy.deepcopy().

What We Learned

FHIR _summary=count is the right way to verify against clinical data without touching PHI. It returns one number. That's it. That's the whole privacy story.

The x402 protocol is simpler than it looks. It's just HTTP — 402 with instructions in the body, then a header on the retry. Any HTTP client, any agent, can implement it in ten lines.

ClickHouse's MergeTree handles append-only alert updates fine for a demo. For production you'd want ReplacingMergeTree so payment_status updates deduplicate properly.

Built With

Share this project:

Updates