FormBuddy

Reads any form, fills your form, remembers you, and waits for your approval.

Comment

Inspiration

In Nepal, a student like Maya (20, Hetauda) fills the same personal details into 10–15 forms a year: ward residency letters, campus scholarships, exam registrations. Each form asks for the same name, citizenship, ward number — in slightly different wording, on different sites. A local cooperative helping 30 families faces this ×30: ~150 minutes of repetitive paperwork per batch, and one guessed field gets the application rejected.

Existing autofill pastes blindly and submits. We wanted the opposite: an agent that reads the real form live, admits what it doesn't know, and never submits without explicit human approval.

What it does

FormBuddy is a Good Neighbor agent with two modes, one brain:

  • Cloud mode — paste any form URL. FormBuddy drives a managed AgentCore Browser to discover fields, matches them against your profile (or a photo of your citizenship/SEE certificate via Bedrock vision), drafts the exact fill plan, and submits only after you click Approve & Submit.
  • Extension mode — open any form in Chrome, click Analyze This Page. The extension reads your tab's HTML (free, no cloud fetch), FormBuddy proposes the plan, Authorize & Fill fills your tab — and you click Submit yourself. Files from the Document Vault auto-attach.

Safety properties: propose_form_fill refuses to save a plan with empty required fields; the agent owns no submission tool; every step (matched → proposed → approved → completed/failed) lands in a durable audit trail; chat "yes" can never approve — only the button click counts.

How we built it

  • Agent: Strands Agents SDK orchestrator + isolated Haiku sub-agents (inspect_form, inspect_provided_html, document_parser, fill_and_submit_form) — structured extraction on cheap models, reasoning where it matters.
  • Approval boundary: propose_form_fill → pending_approval → human decision → resume_after_approval / record_extension_fill_result, with idempotency guards and a Retry path so users never hang on submission_failed.
  • Backend: FastAPI thin wrapper over agent/ (no duplicated logic), serves the web UI from the same origin; session/audit stores are dual-backend (local JSON ↔ DynamoDB) behind env vars.
  • Frontend: zero-build web chat UI + Manifest V3 Chrome extension (side panel, chrome.storage.local profile vault).
  • Cost engineering: Haiku everywhere by default (~$0.25/1M, Sonnet opt-in via env), extension-first demoing ($0 Browser), max 3 live Browser runs, ephemeral Runtime (<30 min, ~$1–2 video day), 48 local tests at $0.

Challenges we ran into

  • Cloud fill flakiness: a Google Forms checkbox click crashed the Playwright context mid-fill. Fixed with selector unescaping, safe single-click + re-init-and-continue prompts, and one fresh retry.
  • Email validation failures: typed values weren't recognized ("This is a required question"). Fixed by always dispatching input+change events and verifying values stuck before submitting.
  • Agent hallucination: the orchestrator forged human approval entries when users typed "yes" in chat. Fixed by refusing actor="human" in the agent's log tool, grounding chat with real session state, and making the button the only approval path.
  • Live Server wipes: demoing on VS Code Live Server (:5500) reloaded the tab on every backend write. Switched to plain http.server + added reload detection in the extension.
  • Terminal UX: 1–2 min synchronous fills looked hung. Moved fills to background threads with live progress polling.

Accomplishments that we're proud of

  • Real end-to-end cloud submission to a live Google Form, verified in Form Responses.
  • Extension fill with vault file auto-attach via DataTransfer — no manual upload clicks.
  • A genuine autonomy boundary: incomplete plans are structurally unproposable, not just politely declined.
  • Full audit trail a cooperative could actually hold accountable.

What we learned

Cost is a design constraint, not an afterthought: model tiering, $0 paths first, and ephemeral cloud shaped every architecture decision. And approval UX is safety UX — every "stuck" state we removed (retry, progress polling, reload warnings) was a trust bug, not a cosmetic one.

What's next for FormBuddy

Nepali-language replies, multi-form batch mode for cooperatives (one profile → 30 submissions with per-family review), AgentCore Memory for cross-device profiles, and allow-listed CORS + rate limiting for production.

Built With

Share this project:

Updates

Submission history