Benefits Check

About the Project

Below is our story—from the spark of inspiration, through what we learned and how we built BenefitsCheck, to the obstacles we overcame along the way. Mathematical notation is rendered with LaTeX where helpful.

Inspiration

• Anthropic Conversation We kicked off by talking to Michael Lai, Head of AI in Governance at Anthropic, who shared firsthand how policy teams struggle to keep up with evolving benefit‐eligibility rules.
• 3branches.org The lucid presentation of the U.S. federal system on 3branches.org inspired us to think: How might we similarly demystify the bureaucracy of benefits?
• Challenge Prompt We discovered the hackathon’s prompt doc describing “democratizing access to government services” and immediately saw the potential: let’s make eligibility as simple as checking boxes—literally.

What We Learned

1. Document Processing Is Harder Than It Looks Dealing with PDFs, scans, and photos taught us that real‐world data is noisy. We learned to pre-process images (deskew, denoise) and combine OCR outputs with confidence thresholds.
2. Policy Encoding Requires Care Translating HR1’s text into structured rules required iterating between legal summaries and engineers. We adopted a simple logical form:

$$ \text{Eligible} \; \Longleftrightarrow \; \bigwedge_{i=1}^n \text{Requirement}_i
$$

where each \$\text{Requirement}_i\$ corresponds to a document check (e.g., income, residency).

1. RAG Performance Matters We experimented with Retrieval-Augmented Generation. We found that using a hybrid similarity score

$$ S(u,v) = \alpha\,\cos(u,v)\;+\;(1-\alpha)\,\frac{|u\cap v|}{|u\cup v|}
$$

(with \$\alpha=0.7\$) gave the best balance between semantic and keyword matches.

How We Built It

1. Frontend

• Plain, Intuitive UI: A single‐page React app where users drag & drop documents—no sign-up needed.

• Real‐Time Feedback: Axios + WebSockets to show extraction progress.

  1. Backend

• LangGraph.js + Gemini 2.5 Pro: We used LangGraph to parse unstructured text into JSON, then sent policy‐matching prompts to Gemini.

• RAG Layer: A local vector store (Pinecone) indexes federal and state policy PDFs. We retrieve top-k passages for context.

• In-Memory Privacy: All file data stays in RAM; nothing is persisted to disk.

  1. Deployment

• Dockerized microservices on AWS Fargate behind an Application Load Balancer.

• Auto-scaling rules trigger at 50% CPU, ensuring responsiveness under load.

Challenges Faced

• Accessing Federal Documents Many benefit manuals are buried in PDF repositories with inconsistent naming. We automated a crawler but still had to manually verify ~20 documents for each program.
• Policy Ambiguities Local ordinances (e.g., Healthy San Francisco) sometimes conflict with federal rules. Resolving these required building a small “precedence” engine to choose which rule applies.
• RAG Tuning Early on, our RAG chain hallucinated eligibility outcomes. We countered by adding a verification step: if the LLM’s answer confidence $p < 0.8$, we fall back to a conservative “manual review” recommendation.
• Performance vs. Privacy In-memory processing limited how much caching we could do. We optimized by batch-tokenizing documents and reusing embedding computations across sessions.

This journey taught us not only the technical intricacies of document AI and LLM chaining, but also the human importance of making government support accessible. BenefitsCheck is our answer to closing the gap between eligibility and enrollment—because nobody should miss out on the benefits they deserve.

Built With

• neon
• next.js
• prisma
• react
• typescript