Setu — a Referral Copilot you can actually trust

Inspiration

In India, when a patient or a care coordinator needs to decide where to go for dialysis, an emergency, or a delivery, the decision often rests on word of mouth or on unverified online listings. A facility can claim almost anything — every clinic's website says it does everything.

We were handed 10,000 messy facility records and asked a deceptively simple question: can a non-technical planner trust this data enough to route a real patient? That reframing — not "is the data clean?" but "is it trustworthy enough to act on?" — shaped the entire project. A referral tool that confidently sends someone to a facility that can't actually help is worse than no tool at all. So we built one that is honest about what it knows, what it doesn't, and why.

What it does

Setu takes a plain-language need and a location — "dialysis near Jaipur", "emergency surgery near Patna" — and returns an evidence-attached shortlist of facilities.

For every candidate it shows:

Matching evidence — the exact sentence from the facility's own text that supports the capability ("Dialysis unit with 10 machines"), with its source field and links, so nothing is a black box.
Missing or suspicious evidence — capabilities a facility claims but can't back up, surfaced rather than hidden.
Three readiness scores — Authenticity (is it real and reachable?), Clinical matchability (can we tell what care it provides?), and Geographic usability (can we place it to route?).
District need context — drawn from national health survey data, so a planner sees not just supply but local demand.

Crucially, when the evidence is thin, the app says so — it attaches a "limited evidence, verify before referral" caveat instead of pretending.

How we built it

Everything runs on Databricks Free Edition — PySpark for the data layer, Unity Catalog tables (workspace.default.*) as the contract between analysis and app, and a Databricks App for the UI.

The data pipeline, as a notebook of ~50 cells, flows in stages:

Profiling & a data-readiness scorecard — framed around referral trustworthiness, not generic EDA.
The join chain facility → district → national health indicators, the make-or-break dependency.
Trust signals — a verification score, claim-inflation detection, and cross-field corroboration.
A controlled capability vocabulary — collapsing 2,900+ inconsistent free-text tokens into 33 canonical specialties.
Evidence-snippet extraction — quotable citations graded Strong (operational proof) vs Partial (mention only).
A need → capability synonym layer — so "kidney failure" resolves to Dialysis before search.
Need-aware routing — turning the health-survey demand layer into a demand-vs-supply gap signal.
Three-score readiness + a referral-ready funnel that quantifies honest uncertainty.

The district need score, for example, converts each relevant indicator into a cross-district percentile and direction-adjusts it (low institutional-birth % = high maternity need):

$$\text{need}{d,c} \;=\; \frac{1}{|I_c|}\sum{i \in I_c} \begin{cases} p_i(d) & \text{if higher} = \text{more need} \ 100 - p_i(d) & \text{if lower} = \text{more need} \end{cases}$$

where $p_i(d)$ is the percentile rank of district $d$ on indicator $i$, and $I_c$ is the indicator set for capability $c$.

The app reads a handful of clean tables — facility_capabilities, facility_evidence, evidence_summary, need_synonyms, district_capability_need, district_capability_gap, and facility_scorecard — so the front end never touches raw text.

Challenges we ran into

District names don't match. The national survey and the postal directory disagree because of renames and transliteration — Allahabad → Prayagraj, Bangalore → Bengaluru, Belgaum → Belagavi. A naive join matched only 87% of districts. We built a curated 90-row alias crosswalk and lifted it to 99.9%.
Free-text capabilities are chaos. 2,900+ specialty tokens in mixed camelCase and plain English. Mapping them to a clean vocabulary — while disambiguating collisions like orthopedic vs orthodontic — took careful keyword design and got us to ~99% coverage.
Telling genuine capability from marketing. A clinic claiming 40 specialties is almost certainly scraped advertising. We had to corroborate claims against the facility's own procedure/equipment text rather than trust the specialty list.
Honest data quirks. The survey encodes small-sample estimates as parenthesized values like (27.0), which silently become null if you don't clean them first.
Resisting the easy lie. The biggest conceptual challenge was not conflating "looks legitimate online" with "good for a referral." They're different questions, and the data tempts you to merge them.

Accomplishments that we're proud of

An evidence funnel that's honest by construction: ~99% of facilities can appear, but only ~64% can show citable evidence and ~52% have strong operational proof — so ~35% carry a "limited evidence" caveat instead of false confidence.
Splitting one vague "trust" number into three independent scores, which exposed that ~29% of facilities look trustworthy online but are clinically thin — the exact rows a referral tool must down-rank.
Need-aware routing using the national health survey — a layer most teams treat as decoration — to flag districts with high need and thin verified supply.

What we learned

The hard part of a referral copilot isn't the matching — it's trust and honest uncertainty. Clean data and a working join are table stakes; the value is in being able to cite why a recommendation is made and to admit when the evidence is weak. We also learned, concretely, that digital presence is a poor proxy for clinical usefulness — a lesson that changed how we score facilities.

What's next

Location → coordinates using the postal directory as a gazetteer, so every candidate can show a real distance.
A gold-label validation set — manually checking ~25 facilities against their live websites — to report calibrated accuracy on our "Strong evidence" label.
Persistence for coordinators: saved shortlists, notes, and overrides, so the app remembers the work.