About the Project — Compass

Inspiration

Millions of Californians (and ~40 million Americans nationwide) qualify for aid yet never finish the paperwork. Surveys show 47% find government applications “overly complex,” and >$140 B in benefits go unclaimed every year.¹
We watched friends juggle five portals, print 30-page PDFs, and re-type the same SSN half a dozen times. Compass was born to shrink that gap: one friendly conversation that figures out every program you qualify for and hands you finished applications—no clipboard, no jargon, no wait-list.

What It Does

Compass is a voice-first digital case-worker for public benefits (CalFresh, Medi-Cal, Alameda County IHSS today). Right now, we focus on public benefits assistance programs for those in need, helping low-income families, the elderly, college students, and more figure out which assistance programs they qualify for and quickly complete all of the applications at once.

Conversational Agent (Vapi + Claude 4)
Greets the user, gains consent, and runs an adaptive screening interview.
Memory & Eligibility Agent (Letta)
Stores answers, matches them against a JSON rule catalog, returns eligible programs and any still-missing fields. Creative Use - Letta replaces Redis/Postgres, acting both as the long-term memory layer and short-term retrieval layer per person, updated automatically through conversation history and without schemas
Document-Filling Agent (Claude 4 PDF prompt)
Maps normalized field names → PDF / HTML IDs, fills every field (checkboxes included), and returns ready-to-submit files.

Result: Speak once → receive fully populated PDFs with zero duplicate questions even when multiple programs need the same data.

How We Built It

Vapi orchestrates the conversation flow with multi-step nodes powered by Anthropic's Claude 4
LLMs (Anthropic's Claude Opus 4) handle natural language + validation and guide Q&A; system prompts enforce accessible, welcoming tone.
Letta parses the catalog’s eligibility requirements to return matching programs, determines what information we still need to complete relevant applications, and stores memory about the user.
Form filler uses Claude Opus 4 to automatically fill out PDF forms, matching user data to text and checkbox inputs.
Front-end Built in Next.js, using Vapi for real-time speech transcription. Dashboards for easy access and navigation. We built everything in less than 24 hours!! 🏃💨

Challenges We Faced

Mapping 50+ form fields to conversation-friendly variable names without losing fidelity and without duplicating questions to the user.
Validating ambiguous answers (“uh… maybe monthly?”) and re-asking the user politely.
Cross-agent memory sync between Vapi and Letta.
Checkbox hell: we spent two hours getting our form-filler to accurately interact with checkboxes!
Rate-limits and exhausted credits at 4 am, forcing us to come up with creative ways to keep moving.

Accomplishments We’re Proud Of

End-to-end demo: speak to Compass → instant, fully populated CalFresh PDF.
Zero duplicate questions, even when multiple programs need the same data.
Plug-and-play program catalogs (including program name, description, eligibility requirements, and application requirements) are stored in a JSON file after LLM processing/information extraction, so adding new programs is a single file change.
Accessibility: works over a low-data phone call or web interface.

What We Learned

Prompt design for structured extraction is half the battle, but still is super important for guiding LLM-enabled functionality.
Users love hearing “let me fill that out for you” instead of “go to this portal”.
PDFs are relics, but they’re not going away soon in government. Automation bridges this gap!

What’s Next

Expand catalog to more assistance programs housing vouchers, LIHEAP, child-care subsidies, to more geographies (counties/states), to more domains (construction permitting, procurement contracting, etc.)
Multilingual flows with automatic language detection to expand language access, particularly for immigrants where English may not be their first language
SMS fallback for alternate ways to interact with the agent.
Document upload & OCR to attach pay stubs or IDs.
Caseworker dashboard so humans (whether a caseworker, secretary, or college student) can more easily override or approve submissions.