HaqSetu - Bridge for Rights

The pension is hers. The documents are the wall. HaqSetu is the bridge — she speaks, she snaps a photo, and gets back the benefits she's owed, already filled out and ready to file.

Comment

💡 Inspiration

I built HaqSetu for the poor people of rural India — the farmers, widows, daily-wage workers, and first-generation students for whom a missed pension or an unclaimed scholarship isn't an inconvenience. It's the difference between eating and not, between a child staying in school and dropping out.

In India, the money and services meant for exactly these people are real, budgeted, and waiting — pensions, scholarships, food rations, health cover, wage guarantees. And every year, an enormous share of it is never claimed.

Not because people are ineligible. Because the benefit arrives as a form they cannot read, in a language that isn't theirs, behind a process no one ever explained.

I kept coming back to one person — call her Sunita: a widow with two children and no land, legally entitled to a widow's pension, a girl-child scholarship, and subsidized rations. She receives none of them. She doesn't know the first exists, can't read the second, and a local middleman offered to "arrange" the third — for a cut of her own entitlement.

That is the injustice that started this: a right becomes a bribe the moment someone else has to read the paper for you. I didn't want to build another app that tells people they "might be eligible." I wanted to build the bridge that carries them the rest of the way — to the filled form, ready to file.

North Star: No one should lose what they are legally owed simply because they cannot read a form.

🌉 What it does

HaqSetu turns a spoken sentence and a photographed document — in the user's own language — into a single deliverable: the Claim Dossier.

  1. 📄 "What this paper means" — any official document the user photographs is decoded in their language: what it is, what it's asking, the deadline, and whether it's an opportunity, a debt, or a scam.
  2. 💰 "What you're owed but not claiming" — a ranked list of every benefit they qualify for, each with a plain-language "why you qualify" that cites the exact government rule it matched.
  3. ✅ "Your ready-to-file paperwork" — the completed official forms, a "bring these" checklist, and where and by when to submit.

The conversation is only the door in. The output is a verifiable packet — which is exactly why HaqSetu is not a chatbot.

🏗️ How I built it

The core design decision is architectural: the language model lives at the edges, and a deterministic rules engine lives at the core. The model does what only a model can — understand messy speech, read a creased document, explain a rule in someone's mother tongue. It is never trusted to decide who is eligible.

 VOICE ─┐
        ├─▶ [1] INTAKE        speech / text → provenance-carrying facts
 PHOTO ─┘
              [2] DECODER      document photo → meaning, deadline, risk   (vision)
              [3] ENTITLEMENT  one agent per scheme, concurrently         (code decides; model explains)
              [4] ACTION       fills the real form, field by field        (model selects; code fills)
              [5] VERIFIER     every claim cites its rule; every field its fact
                    ▼
              📦 THE CLAIM DOSSIER

Stack: TypeScript + Express, a multi-agent pipeline (intake → decoder → entitlement → action → verifier), pdf-lib to fill real government form PDFs, and zod schemas enforcing structured output at every model boundary so a malformed response can never leak downstream. Built end-to-end with OpenAI Codex driving GPT-5.6.

The multimodal + multi-agent + tool-calling trio does the heavy lifting:

  • Multimodal reads the user's actual documents by photo and serves the non-literate user by voice.
  • Multi-agent orchestration fans out one agent per scheme, concurrently — a genuine parallel system, not a single mega-prompt.
  • Programmatic tool calling drives the form-filling: the model selects which PDF fields have a backing fact, and code fills the value — the model is structurally forbidden from inventing one.

The Verifier is the moat

Every scheme \( s \) has a rule set \( R_s \), and eligibility is a pure conjunction of checkable predicates over the citizen profile \( p \):

$$ E_s(p) = \bigwedge_{r \in R_s} r(p), \qquad r(p) \in {\text{true}, \text{false}} $$

For example, an income-tested pension is simply

$$ r_{\text{income}}(p) = \big[\, I(p) \le \tau_s \,\big], $$

where \( I(p) \) is the citizen's annual income and \( \tau_s \) the scheme's threshold. Because these are ordinary boolean programs — not model opinions — the result is reproducible and auditable.

The Verifier then enforces one hard invariant before anything reaches the user:

$$ \forall e \in \text{Dossier} : \operatorname{cite}(e) \neq \varnothing $$

No entitlement is ever shown without a matched rule; no form field is ever filled without a source fact. In a domain where a hallucinated "you qualify" could send someone in a remote village on a bus-ride to an office only to be turned away and humiliated, this isn't a nicety — it's the whole point.

To prove it isn't hand-waving, the widow-pension scheme uses the real eligibility rules published on the Uttar Pradesh SSPY pension portal (sspy-up.gov.in) — widow, age \( \ge 18 \), annual family income \( \le \) ₹2,00,000, state resident — and the dossier cites that source on screen.

📚 What I learned

  • The output is the product, not the conversation. The hardest discipline was resisting "just make it a chatbot." Judging every feature by "does this get a real person closer to money they're owed?" killed a lot of clever ideas and saved the project.
  • Trust is an architecture, not a disclaimer. You cannot prompt your way to "never hallucinate eligibility." You have to make it structurally impossible — which is why the rules engine and the citation invariant exist.
  • Constrain the model, don't beg it. Forcing structured output with schemas, and using tool-calling to select rather than generate form values, turned a flaky demo into a reliable system.
  • Depth beats breadth. Covering a handful of real schemes end-to-end — correctly, with cited rules — is far more convincing, and more honest, than claiming to cover five hundred.

🧗 Challenges I faced

  • Making hallucination impossible, not just rare. The Verifier invariant went through several iterations before "no citation ⇒ not shown" became a true hard gate instead of a soft filter.
  • Filling real government PDFs. Mapping loosely-extracted facts onto rigid, inconsistently-named form fields — without ever fabricating a value, and without crashing on non-Latin (Devanagari) names — was fiddlier than any AI part of the build.
  • Structured output under pressure. Getting reliable, schema-valid JSON out of a multimodal model for every document, every time, drove the decision to validate at every boundary and fail loud rather than pass bad data on.
  • Language and literacy as first-class constraints, not settings. Designing so the conversation is optional and the dossier is the deliverable meant rethinking the whole flow around a rural user who may not read at all, and may share one basic smartphone for an entire household.
  • Building rigorously under a hard resource limit. With model quota to ration, I leaned on tests that mock the model (so they cost nothing) to catch problems before spending a live call, and kept the model client thin and configurable to spend as little as possible. Building something this ambitious under real constraints was the toughest — and most honest — part of the whole project.

🎯 Why it matters

This is not a convenience app. For these families, an unclaimed ₹1,000-a-month pension or a missed scholarship is the line between dignity and desperation. The state is digitizing faster than its poorest citizens can follow, complexity compounds every year, and the middleman economy grows fat on the gap. HaqSetu closes that gap with the one thing that scales where fieldworkers can't: an AI that reads for you, proves why you qualify, and hands you the filled paper — in your own voice, in your own language.

HaqSetu — from paper you can't read to the rights you're owed. Built for the rural poor. Not advice. Not a chatbot. A bridge.

Built With

Share this project:

Updates