Inspiration

Every Pakistani shopper has stared at a bill and thought "this doesn't add up." A quiet 10% service charge, a packaging fee nobody mentioned, an item rung up twice, a faded thermal receipt that is unreadable a week later. Checking all this by hand is tedious, so most people just don't. And when friends split a restaurant bill, someone always ends up overpaying.

What it does

SpendTrace audits a receipt directly on the photo. You upload an image, and instead of a chatbot summary you get the receipt itself with colored overlays:

Green: consistent
Yellow: needs your review
Red: audit finding (wrong total, wrong tax, duplicate item, hidden charge)

Key features:

Capture Fix: straightens tilted photos, removes shadows, and sharpens text.
Audit Engine: checks item sum vs subtotal, tax, and the grand total, and flags duplicates and unusual charges, with the arithmetic shown.
Verify Layer: low-confidence numbers turn yellow. Click one to see the original crop, correct it, and watch the audit recalculate live.
Faded Receipt Rescue: image AI restores faded thermal receipts with a before/after slider. Regions that are still unreadable are marked "unreadable", never guessed.
Tap to Split: tap items to assign people, and tax and service are shared proportionally.

Every finding links to its exact region on the image. Uncertainty is always visible and never silently corrected.

How we built it

Backend: Python 3.11, FastAPI, SQLite
Vision pipeline: OpenCV (perspective correction, shadow removal), Tesseract for OCR with word-level confidence
Audit engine: plain deterministic code using decimal arithmetic with a ~1 rupee rounding tolerance, so there are no AI guesses in the math
Generative AI: the Gemini API for receipt restoration and a "second opinion" read of low-confidence crops, behind a single gate file that enforces a daily request cap, caching, one attempt with no retries, and JSON validation
Frontend: vanilla JS with an SVG overlay, no build step
Testing: synthetic receipts with known ground truth (wrong totals, duplicates, wrong tax) in tilt, blur, shadow and fade variants

Challenges

Trusting AI output. Generative models can "fix" a number into something plausible but wrong. We made a rule: AI suggestions are a separate unverified layer, never overwrite a value, and the user accepts or rejects each one. If the AI disagrees with local OCR, the item stays yellow.
OCR on real receipts. Faded thermal paper and low-light phone photos broke our first version. [Describe one specific problem you hit and how you fixed it.]
Budget limits. We capped API calls per day and batched all uncertain crops into one request using a contact sheet.

Results

On [N] synthetic test receipts: [X]% of fields read correctly, [Y] of [Z] injected errors caught, [W] false alarms. Word readability on faded receipts improved from [A]% to [B]% after restoration.

What we learned

[1-2 honest sentences, e.g. that showing uncertainty builds more trust than hiding it, or what surprised you about OCR.]

What's next

Tamper detection for edited digits, price-drift tracking across receipts, warranty and return-window reminders, a monthly hidden-charges report, and better Urdu digit support.

All results are labeled preliminary. SpendTrace is a checking aid, not an official or legal verification.