Cabinet Clear

confusion matrix
Grad-CAM results

Inspiration

A few months ago, a routine blood test showed my blood sugar was higher than it should be. My doctor said it might just be because I'd eaten breakfast that morning, so I didn't think much of it. Then it happened again. He told me to come back the next day fasting, for a real test this time.

That night was one of the worst nights of my life. I remember genuinely thinking that if I just willed my body to behave, the next test would come back normal — like discipline could undo biology overnight. I barely slept. The next morning I went through a full round of tests, and the result came back: prediabetes.

Then I was handed medication, and a whole new kind of confusion started. Nobody sat down and explained what the medication actually did to my body, why this specific dose, what I was supposed to watch for, or what would happen if I missed a day. I was a stressed teenager holding a pharmacy bag, trying to piece together what I was now supposed to do for the rest of my life from a label that assumed I already knew.

I'm someone who can read medical terminology, who has access to research papers, who can ask an AI to explain a drug interaction. And it was still overwhelming. That made me think about my own house. I have three younger siblings, and our cabinet is genuinely full of bottles — old antibiotics from someone's ear infection two years ago, fever medicine, things prescribed for one sibling that might or might not still be relevant for another. My parents are busy and, frankly, just as confused as I was.

So this isn't a hypothetical parent I imagined for a hackathon brief. It's the cabinet in my own kitchen. That gap between what medical documentation assumes you understand and what an exhausted family actually understands in the moment is what Cabinet Clear is built to close.

What it does

Cabinet Clear reads a hospital discharge letter — pasted as text, uploaded as a PDF, or photographed — and turns it into a plain-language summary: diagnosis, medications, follow-up appointments, and warning signs. Then it scans photos of the user's actual medicine cabinet, checks expiration dates, and cross-references everything against the letter.

The output is a simple, actionable plan, sorted into four buckets:

Keep and take as directed
Dispose (expired)
Unclear — ask your pharmacist
Missing — pick up from the pharmacy

Every family with an aging parent, a kid on a new medication, or just a cabinet nobody's sorted in years has this exact problem — and right now the only solution is "call the pharmacist and hope they have time." Cabinet Clear turns a 20-minute confused phone call into a 2-minute photo scan. That's the wedge: not replacing pharmacists, just closing the gap between "I got discharged" and "I can actually ask the right question."

Users can also export the whole plan as a PDF to hand to a pharmacist, and the app installs directly to a phone home screen as a PWA. Signing in (bcrypt-hashed passwords, never stored in plain text) saves a history of past discharge letters and cabinet checks, so a family can look back over time.

How we built it

The system has three layers, each with a clearly scoped job:

A custom-trained CNN clarity gate. Before any label is read, a ResNet18 model (trained from scratch via transfer learning on 1,748 images, with Grad-CAM explainability) checks whether a photo is clear enough to trust. If not, it's rejected immediately with a visual explanation of what part of the image failed.
AI extraction. The Claude API reads the discharge letter and medicine labels, turning messy real-world documents into structured data with a self-reported confidence rating per extraction.
A deterministic cross-reference engine. The AI never makes the final keep/dispose call. A transparent, auditable Python rules engine — using real OpenFDA brand-to-generic drug data and fuzzy matching for OCR typos — decides which bucket a medication falls into. We split it this way on purpose: a hallucinated "this is safe to keep" from a language model is a far worse failure mode than a rules engine making a traceable, explainable decision.

Stack: FastAPI (Python), PyTorch/torchvision, Grad-CAM, thefuzz, fpdf2, SQLite with bcrypt-hashed passwords, and a vanilla HTML/CSS/JS PWA frontend.

Challenges we ran into

Deciding where the "decision" lives. Our first instinct was to let the AI read both documents and just tell us what to do. We changed course once we realized that put a safety-critical decision inside a system that can hallucinate.
OCR reliability on real photos. Expiration dates are often on a different panel than the drug name, in low-contrast print — exactly why the CNN clarity gate exists.
Being honest about our model's evaluation. Our CNN clarity classifier scores 100% on its validation set, but that reflects testing on augmented variants of the same source photos, not an independent real-world test set. We disclosed this directly rather than presenting the number without context.
PDF generation with Unicode. Checkmark and warning symbols crashed FPDF's core font encoding — fixed by sanitizing all dynamic text before the PDF layer.

Accomplishments we're proud of

A working three-layer pipeline where each layer's role is intentional, not accidental. A trained, explainable computer vision model with Grad-CAM verification, not just an API call wrapped in a UI. Real FDA drug data integration instead of a hardcoded list. A full account system with bcrypt-hashed passwords and history. A clinical-safety-first architecture where the part most likely to be wrong (the AI) never makes the final call.

We also built this knowing it has to work for someone who is scared, tired, and not a medical professional — a much harder design bar than "works for a developer testing it," and one we think most health AI tools skip.

What we learned

How to separate language understanding from decision-making in an AI system, and why that separation matters more in a health-adjacent context than almost anywhere else. We also learned that a model's validation accuracy is only meaningful in the context of how the validation set was built, which is why we report our CNN's limitations explicitly rather than leading with the headline number.

What's next for Cabinet Clear

Near-term: Expanding the FDA drug database beyond our initial pull, and testing the clarity classifier against independently collected (not augmented) blurry photos — the real-world test our current validation number doesn't yet prove.

Who this is for: Families managing medications for kids, aging parents, or anyone juggling more than one prescriber — the people currently relying on memory, sticky notes, or a rushed pharmacist call. Just as important: pharmacies and clinics themselves, who spend real staff time on medication reconciliation calls that Cabinet Clear could pre-sort before the patient even walks in.

How this grows: Free for individual families — the PWA install costs nothing and needs no app store. Revenue comes from B2B: licensing the extraction and rules engine to independent pharmacies and home health agencies as a white-labeled medication reconciliation tool, where a single avoided callback or ER visit pays for a year of the tool many times over.

Why now: Every discharge letter already exists as unstructured text or a photo — we're not asking hospitals to change any workflow to adopt this, which is usually where health tech dies. We're reading what's already being handed to patients.

What sets this apart: Searching a symptom or drug name online gives generic information that isn't matched to your actual letter or your actual cabinet. Cabinet Clear's advantage isn't more information — it's information that's already been cross-referenced against what you were specifically prescribed, so the answer is a decision, not another search result.