PaisaPilot — your money copilot

Inspiration

Money apps make you do the work: open the app, find the category, type the amount, add the note. For a student in India juggling UPI payments across five different apps, that friction kills the budget by week two. And deal-hunting lives in yet another app entirely — price alerts, wishlists, tabs you never check.

What if one voice-first agent did both? No forms, no typing: just talk, and stay in control of your money.

What it does

PaisaPilot is a simulated Alexa+ experience — a money copilot you talk to:

• Voice expense logging — “Alexa, I spent 250 on lunch.” The agent logs it, auto-detects the category (food, transport, shopping, bills, entertainment, health, education, travel), and talks back. Logging takes under five seconds.
• Budgets that warn you first — “Set budget 5000 for food.” PaisaPilot tracks the month and warns at 80% and 100%: before the money is gone, not after.
• A dashboard that tells the truth — spending by category, budget bars, recent expenses, all live.
• Deal Watch — “Track iPhone 17 below 70000.” PaisaPilot watches the price and fires a voice + visual alert the moment it drops to your target. Budgeting and bargain-hunting, one agent.

How I built it

• Backend: Python Flask + SQLite. agent.py runs the intent pipeline: an Amazon Bedrock hook (llm.py, invoke_model) for LLM intent understanding, with a full offline heuristic fallback — so the demo never depends on a key.
• Frontend: a single-page web app with an Alexa-style voice UI — Web Speech API for speech-to-text, speechSynthesis for replies — plus a Chart.js dashboard (vendored locally, works offline) and a price-watch tab.
• Price checks: best-effort extraction from JSON-LD / meta / ₹ patterns, with manual refresh and check-all.
• Demo pipeline: the entire 57-second narrated demo was captured headlessly — Playwright drives the scripted conversation, screenshots every beat, ffmpeg assembles, TTS narrates. Deterministic, re-runnable.

Challenges I ran into

• Web Speech API is Chromium-only and shaky on some accents — the #1 risk to a voice demo. Mitigation: every voice flow also works by typing, and the TTS replies are pre-generated, not live.
• Price extraction fails on JS-rendered retailer pages (most big Indian stores). Shipped with JSON-LD/meta parsing + manual price entry; a headless-browser fetcher is the planned upgrade.
• Demo determinism. A live voice demo is a chaos vector, so I scripted and screenshotted the whole flow headlessly instead of hoping for the best on stage.

Accomplishments that I'm proud of

• A working voice-first money agent, end to end, in a single day — from idea to submitted hackathon entry in one morning.
• Submitted three weeks early, with a public repo (MIT), a narrated demo video, and a friction log kept from the first command.
• The agent genuinely works: heuristic parsing handles real phrasings (“I spent ₹500 for groceries”), budgets warn at exactly the right moments, and the whole thing runs offline.

What I learned

• Voice beats forms for habit-forming tools. If logging an expense takes more than five seconds, the habit dies. Talking is the fastest input there is.
• Always ship an offline fallback for demo-critical paths. The Bedrock hook is the future; the heuristic parser is the present that never fails on stage.
• Friction logs are a feature, not a chore — writing down every snag made the build calmer and earned judging bonus points.

What's next for PaisaPilot

• Wire up Bedrock fully once the $150 hackathon AWS credits land — then enter the AWS Builder mini-challenge and update this submission.
• Headless-browser price fetcher for JS-rendered stores (Flipkart, Amazon.in).
• Real Alexa+ skill path — graduate the simulation into an MCP server / Agent Skill once the Alexa+ preview opens up.
• UPI autopilot — the dream: read-only SMS parsing to log spends automatically, with the user's explicit permission. Talk first, automate later.

Built With

• flask
• javascript
• python
• sqlite
• web-speech-api