Shelf Aware

counting number of drinks left

Inspiration

We kept coming back to the same dumb realization: it's 2026 and buying stuff still starts with a human noticing something. You run out of coffee, you remember you ran out of coffee, you open an app, you tap through checkout. Every "agentic commerce" demo we'd seen just moved the chatbot around. So we asked a weirder question - what if the shelf itself could just... handle it? No app. No prompt. The environment is the interface.

What it does

Shelf Aware points a camera at your shelf and watches it. A local vision model counts what's actually there, and the moment stock dips below a threshold you set, an agent goes out, finds the item, and reorders it. You literally do nothing. You take the last coffee bag off the shelf, and somewhere, an order is already happening.

How we built it

A camera, on-device computer vision, per-item thresholds, and a browser agent that does the actual purchasing - glued together during the Saturday. The best part: everything runs locally. No cloud vision APIs, no API keys, no images of your kitchen going anywhere.

Challenges we ran into

Turns out counting objects on a shelf is way harder than it sounds. Lighting shifts, weird angles, items hiding behind other items, and products that look nearly identical all fought us constantly. And once the ordering worked, we had a new nightmare: making sure the agent didn't get trigger-happy and buy the same thing five times. Debouncing a purchase is a very different vibe than debouncing a button.

Accomplishments that we're proud of

We got the full loop working end-to-end: a physical change on a real shelf becomes a real commerce action, with zero taps in between. And the architecture doesn't care what it's watching - office supplies, a medicine cabinet, retail shelves, warehouse bins. Point the camera, set a threshold, done.

What we learned

Agentic commerce doesn't have to start with a conversation. Cameras and sensors can express intent more honestly than any prompt - the empty shelf is the request. And keeping the models local isn't just a privacy win; it's what makes ambient systems like this cheap enough to actually deploy everywhere.

What's next for Shelf Aware

Better recognition, approval flows and spending caps (so the agent can't go rogue with your credit card), multi-shelf and multi-supplier support, and squeezing the whole thing onto a Raspberry Pi so any shelf, anywhere, can become aware.

Shelf Aware: the shelf notices, decides, and reorders — because the best interface for agentic commerce is no interface at all.