BioPaws AI — Microchip-Free Biometric Pet ID

Inspiration

Every year millions of pets get lost. The classic fix — a microchip — only works if a vet finds the chip, a scanner is on hand, and the owner's info is still current in a registry. That's three failure points before a reunion ever happens. Meanwhile, if you find a stray on the street, you can't do anything useful in the moment: no chip, no ID tag, no idea who it belongs to.

We wanted the thing you'd actually use on the street: point your phone's camera at a stray, get a biometric fingerprint instantly, and check it against a registry of missing pets. No app download, no hardware, no vet visit. The pet's face and nose print are the chip — carried with the animal, impossible to lose.

What it does

BioPaws is a mobile web app that identifies pets by biometrics instead of microchips.

• Registration: a shelter or owner uploads a photo of a pet (and optionally a recording of its bark/meow). The app extracts a 1024-dim MobileNet embedding of the face/nose area plus a 52-dim acoustic fingerprint of the voice.
• Finding: someone scans a stray with their phone camera. The app computes the same fingerprint and searches the registry of missing pets.
• Matching: a fused 1076-dim vector is compared by cosine similarity. A genuine match confirms the pet and shows the owner's contact so the finder can reunite them in minutes, not days.

How we built it

Stack. React + Vite + IndexedDB, Tailwind CSS, TensorFlow.js (MobileNet v1, alpha 1.0) running fully in the browser. The vector registry is local-first today but mirrors a server schema so it can scale to a cloud registry without a rewrite.

The biometric pipeline. Image features come from MobileNet's pooled content embedding (L2-normalized, classifier removed). Audio features are a waveform fingerprint built from an RMS energy envelope, zero-crossing counts, and amplitude stats. The two are fused into one vector:

$$v = [\text{image}{1024}, \text{audio}{52}] \quad \Rightarrow \quad \text{dim} = 1076$$

Matching is cosine similarity, fused per modality:

$$sim(a,b) = \frac{\sum_{i} a_i b_i}{\sqrt{\sum_i a_i^2}\sqrt{\sum_i b_i^2}}, \qquad \text{score} = 0.7\cdot sim_{\text{image}} + 0.3\cdot sim_{\text{audio}}$$

Calibration on real data, not assumptions. We measured the whole system against real media — actual dogs and cats from Wikimedia Commons, including genuine same-animal multi-shot pairs (Bo the Obama dog, Grumpy Cat), and real bark/meow recordings from archive.org. The 0.72 match threshold sits in a real, measured gap:

pair type	best cosine
same animal, different photo	0.80 – 0.81
different animal, same species	max 0.65
cross-species	max 0.58

That gap is what makes a biometric matcher trustworthy — and it's measured, not guessed. Confidence is derived from score: confidence = min(99, score·55 + 38).

Testing the hard way. We drove real Chrome through a CDP-driven E2E suite: 53/53 real-media scenarios passed — register Bo from photo A, scan photo B → match; scan a different dog → correctly rejected; same for cats; real bark vs. meow audio fused into profiles. We also shipped a one-click Auto Demo that walks the real registration→scan→match→reject pipeline with bundled real photos (~11 seconds, all steps verified).

Challenges we ran into

Our first matcher was fake. v1 used a hand-rolled 64-dim color/layout hash. On real data it collapsed: a distinct dog scored 0.97 — indistinguishable from a self-match. It looked great in demos and was useless in the real world. We threw it out and rebuilt on genuine neural embeddings.

A "better" model made us worse. We fine-tuned an ArcFace muzzle embedding (trained on 4,000 cattle muzzles) expecting a leap in accuracy. In cattle space it scored R@1 100% — but transferred to dog/cat photos it collapsed completely: same-pet and different-pet both scored ≈0.69, so no threshold could separate them. It passed eval on its training domain and failed our star test on real dog/cat media. We reverted, and kept the failure documented instead of buried.

Calibrating an honest threshold. Finding a threshold that rejects different pets while accepting the same pet took real-data measurement — the whole point of the 53/53 suite. The 0.72 line works only because we found the gap.

Browser automation races. The one-click Auto Demo kept failing in ways the app didn't: the Finder module's mount effect reset scan state mid-scan; a stale "ready" label caused matching against the previous photo's descriptor; the registration form reset pet names to Max/Luna in an effect that clobbered our typed input; and our own log output self-polluted result detection. Four subtle state-timing bugs, each fixed by waiting for the right UI signal.

React 19 file input. DataTransfer + change events were flaky/untrusted; we invoke the input's React onChange handler directly with a synthetic event object, which reliably fires every time.

Accomplishments that we're proud of

• 53/53 real-media E2E tests — every claim measured through the actual app, in a real browser, on real photos and audio.
• A threshold with a real gap behind it: same-pet ≥ 0.80, different-pet ≤ 0.65, threshold 0.72.
• Honest engineering: we document what we didn't build — the muzzle model failure is written up, not hidden, and "hero" UI stats are flagged as flavor copy.
• A demo that isn't a demo: the Auto Demo computes genuine pixel-derived embeddings on every run — no canned results.
• A privacy story built for the cloud: embeddings are non-reconstructable by design, and owner contact will be client-side AES-256-GCM encrypted so the registry knows pets, never people.

What we learned

• A generic image embedding is not a biometric. MobileNet's content features separate different-looking animals but they're not muzzle-grade identity features — that's exactly why our fine-tuned model attempt mattered, and why a dog/cat muzzle model trained on real data is the real upgrade path.
• Validate on the deployment domain. Our model was great on its training domain and useless on our real one. Star tests on real media caught what accuracy tables missed.
• The threshold is a measurement, not a number. Picking 0.72 from a spreadsheet would have been guesswork; measuring the same-pet/different-pet gap made it defensible.
• Automation is a mirror for your app's state bugs. Every race the Auto Demo hit revealed a real state-reset or stale-state pattern in the UI itself.
• On-device ML is production-plausible. MobileNet runs in a browser tab, works offline after first load, and keeps biometrics on the user's device — but a lost-pet finder inherently needs a network, so the real architecture is hybrid edge + cloud.

What's next for BioPaws AI

1. Tier-1 cloud registry — pgvector/Qdrant top-k matching over encrypted-at-rest blobs, API keys per shelter, delete API, owner contact encrypted client-side (AES-256-GCM). Zero PII on the server.
2. A real dog/cat muzzle model — Phase 1 training data (shelter-collected or commercial), replacing the generic embedding with a species-specific biometric so the matching gap widens further.
3. Shelter integrations — bulk import of shelter intake photos, API quotas, and a public lost-pet matching pool.
4. Confidential enclaves (Tier 2) — matching moves into a hardware enclave so even the operator can't read vectors; drop-in backend swap, no app rewrite.
5. FHE blind matching (Tier 3) — the long-term goal: matching on fully encrypted vectors.

Built With

• css
• jsx
• rust
• tailwind