Architecture

📄 BillsBuddy — AI Bill Assistant on AWS

BillsBuddy is a serverless, AI-powered assistant that turns messy bills into clear actions.
Upload a PDF, photo, or text → it extracts details, schedules reminders, and even generates a mock payment link.
All orchestrated on AWS, with a friendly Gradio UI.

💡 Inspiration

Missing a payment because a bill was buried in your inbox or snapped as a blurry photo is far too common.
We wanted an AI agent that doesn’t just chat—it does:

read any bill,
extract what matters,
set reminders,
and help you pay.

The goal: a tiny, reliable assistant that turns paperwork into actions.

🚀 What it does

BillsBuddy can:

📑 Understand bills (PDFs / images / text): provider, amount, currency, due date, account number, billing period, notes.
⏰ Create a smart reminder plan (7-day, 3-day, same-day) and export an .ics calendar file.
💳 Generate a payment link (mock/demo) for quick testing flows.
💬 Chat with an AI agent to upload a bill or request actions (“extract this and schedule reminders”).
📊 Keep a session history with cards, tables, and CSV export.

All built end-to-end on AWS: S3, Lambda, API Gateway, Bedrock, DynamoDB.
With a clean Gradio UI for testing and demo.

🛠 How we built it

Frontend

Gradio (Python) with tabs:
- Understand Bill
- Schedule & Calendar
- Pay (Mock)
- History
- Chat with Agent

Backend

Amazon S3 → presigned uploads (billsbuddy-uploads/...pdf|png).
API Gateway + Lambda endpoints:
- POST /tools/presign → return S3 presigned URL.
- POST /tools/extract → Textract + Bedrock to parse bill fields.
- POST /tools/schedule → generate reminder plan and .ics.
- POST /tools/pay → return mock payment link + reference.
- POST /agent/chat → Lambda wrapper around Bedrock Agent (no client creds needed).

AI & Data

OCR: Amazon Textract (multi-page support).
LLM: Amazon Bedrock (Claude 3.5 Sonnet) with structured prompts + JSON validation.
Agent: Bedrock Agent runtime with tool-calling hints.
Database: DynamoDB for bill metadata, reminders, and history.

☁️ BillsBuddy System Architecture

This diagram shows how all AWS components work together:

🔑 Key Components

Web Application → user uploads bills & interacts with agent.
Amazon API Gateway → routes requests to Lambda functions.
Lambda Functions:
- AgentChatFn → Bedrock Claude agent chat.
- ExtractBillFieldsFn → Textract + Bedrock extraction.
- ScheduleRemindersFn → reminder plan generator.
- MockPaymentLinkFn → mock payment flows.
- PresignUploadFn → secure S3 uploads.
Amazon Bedrock → LLM (Claude).
Amazon Textract → OCR for PDFs/images.
Amazon DynamoDB → stores extracted data + reminders.
Amazon S3 → stores uploaded bills.
VPC → secure environment.

🔄 Flow

User uploads a bill or chats via the Web App.
API Gateway forwards request → appropriate Lambda.
File uploads → PresignUploadFn → S3.
Textract + Bedrock analyze → ExtractBillFieldsFn processes.
Extracted details → saved in DynamoDB.
ScheduleRemindersFn creates reminder plan; MockPaymentLinkFn returns demo payment link.
User interacts with results via chat agent backed by Bedrock.

👉 In short: A fully serverless, AI-powered workflow to manage bills, reminders, and payments.

✨ Nice touches

📅 Smart Schedule cards: clear timestamps, not raw JSON.
📥 Calendar export (.ics) for reminders.
📜 History: view old bills, summaries, and CSV export.
🎨 Defensive UI: normalized keys, high-contrast CSS, health badge, detailed error messages.

🧗 Challenges we ran into

Model schema issues → Bedrock Anthropic requires exact JSON; solved with validators.
Local creds for agent chat → fixed by routing chat through Lambda.
Presign 500s & CORS → resolved with minimal IAM + consistent content-type handling.
Multi-page PDFs → added concatenation + fallback.
Field inconsistencies → UI normalizes synonyms and shows OCR preview.
Invisible UI text → enforced high-contrast CSS.

🏆 Accomplishments

✅ Working end-to-end pipeline: upload → extract → schedule → pay (mock) → chat.
✅ Human-friendly UI with cards, calendar, history, CSV.
✅ Robust logging/metrics in CloudWatch.
✅ Extensible to new bill formats with minimal changes.

📚 What we learned

Structured prompts + JSON schema validation are critical for LLM reliability.
Textract quirks need heuristics + LLM post-processing.
API Gateway + Lambda keeps credentials secure.
UX matters: JSON vs. polished cards = prototype vs. product.