Inspiration

We came up with this idea a couple weeks before school started. One of our founders was looking for a backpack, but was struggling a lot on picking the right one. That’s when, when walking around UW, they saw the perfect backpack right in front of them. The only issue was, they were on someone else's back and they were walking super fast in front of them. They lost them in seconds. You see someone wearing a jacket you like, but they disappear before you can ask where it’s from. You try a dish you want to recreate, but don’t know its ingredients. You pick something up at the grocery store and wonder whether it fits your diet or costs less somewhere else. These moments all start with something you see. Yet acting on them means pulling out your phone, describing what’s in front of you, and searching across different apps. We built Glance around one question: what if shopping could start right where you’re looking?

What it does

While Glance is an AR shopping lens designed for Meta Ray-Ban Display glasses, it can be better described as a frictionless shopping for the physical world. It connects product discovery, meal planning, and shopping through three modes:

Glimpse: Scan a product, explore likely matches, compare offers, and purchase eligible items through an integrated checkout.
Grocery: Track your physical cart and running total, compare prices, review nutrition and dietary information, and follow a shopping list. Nothing's more frustrating than getting to the checkout and realizing you spent 100 bucks when you meant to only spend 50. Or buying yogurt for 7 bucks when it’s just 3 bucks down the street. We fix both these problems in the grocery. It totals your cart for you, while you shop, and when you pick up an item it compares it for you.
Gourmet: Scan a prepared dish, explore recipes, and add ingredients to the shared Grocery list. A meal you discover through Gourmet becomes a shopping list in Grocery. A product you discover through Glimpse can move from recognition to checkout in the same interface. Our prototype includes live AI recognition and payments that use Stripe’s sandbox.

How we built it

We built the interface with Next.js, React, and TypeScript, with a shared lens runtime for the browser preview and iOS companion. The Swift companion includes a Meta Wearables Device Access Toolkit integration for camera input, display output, and glasses controls. On the backend, Claude analyzes camera images to identify products, interpret grocery items, and suggest recipes. We validate its structured responses before using them in the interface. Recognition feeds into separate services for product listings, grocery tracking, and recipe planning. Supabase supports accounts, saved preferences, orders, and shopping data. Stripe handles saved payment methods and Visa card payments. Purchase totals are calculated on the server, and payment verification is kept separate from order fulfillment. We also built a Three.js landing experience to introduce Glance and demonstrate its interaction model.

Challenges we ran into

Designing for a small glasses display forced us to rethink familiar shopping interfaces. Product information, navigation, and purchase actions all compete for limited space. Focus and button behavior became especially important because the wearer needs to understand what a control will do before activating it. Camera-based shopping also introduces uncertainty. A frame can contain several products, a brand can be unreadable, and the same grocery item can appear across many frames. We had to account for ambiguous selections, repeated observations, and duplicate cart additions. Checkout brought another set of challenges: preventing duplicate payment attempts, handling stale prices, recovering from failed payments, and showing the difference between a confirmed payment and an accepted order. Keeping camera capture, AI requests, and interface updates responsive across the web and native companion remains a major area of testing and refinement.

Accomplishments that we're proud of

We connected three shopping experiences around shared context. Discovering a meal can lead directly to planning a grocery trip, while discovering a product can lead to comparing offers and checking out. We’re also proud of building beyond the visual demo. The project includes account-backed data, server-priced orders, sandbox payments, payment recovery, and receipts. Those details make the prototype a foundation we can continue developing. Most of all, we created a concrete interaction for an idea we wanted to experience ourselves: seeing something in the world and being able to act on it without wasting time opening our phone.

What we learned

We learned that identifying an object is only the first step. A useful shopping application also needs to communicate uncertainty, preserve context, and give the user clear control over what happens next. We also learned how much interface design changes when the screen sits in the wearer’s field of view. Building across a web app, native companion, AI backend, and payment system taught us to define clear boundaries between what the system sees, what it infers, and what the user authorizes.

What's next for Glance

Our next priorities are smoother display performance and more reliable scanning and navigation. We want to test the complete journey with users and measure how quickly they can move from seeing an item to making an informed decision. From there, we plan to expand verified merchant catalogs, improve price accuracy to be linear with speed, and product matching, and move beyond simulated fulfillment. We also want to refine interests and dietary personalization, and make the transition from recipes to grocery shopping more useful. Our goal is to make Glance a natural part of wearing smart glasses: a shopping AR lens and application that understands the moment and helps you act on it.