Warranty Watchdog

An agent that remembers what you own, understands your warranty, and files a claim when something breaks—using receipts, documents, photos, and voice.

Comment

Inspiration

Somewhere in my house is a blender that's been making a burning smell for three weeks. It's still under warranty. I know this because I checked — once, right after I noticed the smell — and then never followed up, because filing the claim meant finding a receipt from eight months ago, figuring out which drawer/email/photo it was buried in, and writing a formal request to a company that doesn't make it easy. So I did what everyone does with warranty claims: nothing. The warranty will quietly expire, and I'll buy a new blender.

That's not a me problem. Everyone has a version of this — a jacket, a kettle, a laptop charger, something under warranty that broke and never got claimed because the proving part is more annoying than the living without it part. The technology to fix this has existed for a while (vision models that can read a receipt, language models that can draft a formal letter) — nobody had wired it together into something that just remembers for you and acts when it matters, without you having to manage it. That gap is what Warranty Watchdog is.

What it does

Warranty Watchdog is an agent that reads your receipts, remembers what you own, and files the warranty claim for you the moment something breaks — with zero further input required if it's confident.

Step one, once per purchase: photograph a receipt. An Intake Agent uses Gemini's multimodal vision to extract the brand, model, retailer, purchase date, and warranty length, and stores it in a persistent Memory Bank. You never look at it again.

Step two, whenever something actually breaks — weeks or months later: describe what's wrong (text, voice, or a photo of the defect itself). A Claim Agent matches the report against everything in the Memory Bank, checks whether the item is still under warranty, and drafts a formal claim letter. If it's confident and the item is covered, a Filing Agent automatically routes and files the claim — no further clicks. If it's not confident, or the warranty has expired, it says so and hands it back to you instead of guessing. Every decision is logged in a human-readable reasoning trace, so you can see exactly why the agent did what it did.

How we built it

Three agents, one direction of trust. IntakeAgent is the only code path allowed to create a tracked product. ClaimAgent can read products and propose a claim but can't write one or send anything externally. FilingAgent is the only agent allowed to perform the outward-facing action. That separation means a bad Gemini response at the matching step can, at worst, produce a wrong draft — visible and reversible — never a phantom product or a claim that silently goes out the door.

The Memory Bank — the persistent store that lets the agent remember a product for months between the receipt scan and the defect report — is built behind a swappable interface, backed by a simple JSON store for local development and structured to swap to Firestore for production with zero changes anywhere else in the codebase.

Every Gemini call runs behind a client wrapper with a real, fully working mock mode: the entire pipeline runs and is testable end-to-end with zero API key and zero cloud setup. That turned out to be one of the best decisions we made — it meant the UI, the confidence-threshold tuning, and the agent orchestration could all be iterated on instantly, without waiting on network latency or worrying about API quota, and it means anyone can clone the repo and see it work in under a minute.

Stack: Gemini (vision + generation) via the Gemini API / Vertex AI, a Google Agent Framework orchestrating the three agents, FastAPI, and a lightweight HTML/CSS/JS frontend with a live animated pipeline view so you can actually watch which agent is working in real time — deployed on Google Cloud Run.

Challenges we ran into

The hardest problem wasn't calling Gemini — it was deciding when not to trust it. An agent that auto-files every claim it drafts is reckless; an agent that asks for confirmation every time isn't really autonomous. We ended up building a single, named confidence threshold that gates auto-filing versus escalating to the human, and treating that threshold as the one real calibration point in the whole system. Getting it wrong in either direction breaks the pitch — too low and it files nonsense claims, too high and it's just a chatbot with extra steps.

A close second: making sure an expired warranty is correctly declined rather than filed anyway. It's easy to build a demo where everything matches and everything works; it's much more useful — and much more honest — to build one that explicitly refuses to file a claim it shouldn't, and says why.

Accomplishments that we're proud of

Every claim in this project is backed by something we actually ran, not something we assumed would work. We wrote a real test suite, actually started the server, and hit it with real HTTP requests to confirm the full pipeline — receipt in, product tracked, defect reported, claim matched, drafted, and filed — end to end, and in the process caught and fixed two real bugs (a matching-confidence bug that would have mis-escalated claims, and a claim-history duplication bug) before they ever reached a demo.

We're also proud of the architecture holding up under its own rule: the three agents genuinely can't do each other's job, which isn't a comment in the README, it's enforced by what each class is and isn't allowed to call.

What we learned

The most reusable insight: the interesting engineering problem in an autonomous agent usually isn't "can it do the task" — Gemini can read a receipt and draft a letter without much effort. It's "how does it know when not to act on its own," and building that decision explicitly, as a visible, tunable, auditable piece of the system, is what turns a demo into something you'd actually trust to file a real claim on your behalf.

What's next for Warranty Watchdog

  • Real per-brand claims-channel discovery, so filing works for any retailer automatically instead of a fixed set.
  • Firestore-backed Memory Bank for real multi-device, multi-year persistence.
  • Proactive expiry alerts — notifying you that a warranty is about to lapse even before anything has broken, not just reacting to defects.
  • A browser extension that captures e-receipts directly from your inbox, so nothing ever needs to be photographed at all.

Built With

Share this project:

Updates

Submission history