Inspiration
Shopping is still divided between the physical and digital world. We can discover something we love in a store, but if we are not ready to buy it, that experience ends when we walk out. As frequent shoppers, we constantly find clothes we like while window shopping, then go home and have to search for every item again. We started asking ourselves, what if leaving the store did not mean leaving your shopping experience behind? That inspired us to imagine the future of shopping, where you can discover something in the physical world, scan it, take it with you digitally, try it on virtually, and purchase it whenever you are ready.
What it does
Our app connects physical and digital shopping into one continuous experience. Users scan the barcode of clothing they discover in stores and instantly add it to their personal digital collection. Anything spotted while browsing online works the same way through our Chrome extension, so every item lands in one place no matter where it was found. At home, they can enter an immersive AR fitting room where AI allows them to see the clothes they scanned on their own body, switch between pieces, and compare items from different stores. Every saved item remains connected to its online product listing, allowing users to go directly from discovery to virtual try on to purchase without having to search for the product again. When it is time to buy, one checkout completes the purchase across every store an item came from.
How we built it
We built Tags as one shared backend behind two connected surfaces, a Chrome extension for adding items while browsing online and a mobile app for scanning tags in physical stores, so every item lands in the same collection no matter where it came from. We use a real time generative video model that powers the fitting room, turning a live camera feed and a garment image into a video of you wearing that item, which let us skip building the full 3D body model we originally planned. An agent built on Grok handles checkout, visiting each store in your collection and completing the purchase so one tap covers items from several stores, with payment settling in USDC through Solana Pay. We built it all during the hackathon using Claude for the coding, Gemini and OpenAI for planning, and GitHub to keep four people working on one codebase.
Challenges we ran into
One of our biggest challenges was connecting physical products with changing online inventory. During testing, one of our barcodes stopped working because the shirt had sold out online, even though we still had the physical item with us. Our original plan also involved creating a full 3D model for the fitting experience, but we quickly discovered that the approach was not working the way we envisioned. We pivoted to an AI and AR based experience that still gave us the immersive interaction we wanted. And somewhere between debugging barcodes and building the future of shopping, we may have had a little too much fun.
Accomplishments that we're proud of
We are proud of pivoting quickly once the 3D fitting idea stopped working and landing on a fitting room built on live video that felt immersive and alive. We are proud of getting two separate ways to add an item, a barcode scan in store and a Chrome extension online, to land in the same collection, of getting a real checkout working end to end through Solana, from a QR code through a confirmed payment, and of getting an agent that completes a purchase across multiple stores from a single tap.
What we learned
We learned that the future of retail does not have to exist entirely online or entirely inside a physical store. The most exciting possibilities come from connecting the two. Building this project taught us how physical products can interact with digital inventory, how AI and AR can create immersive shopping experiences, and how quickly a team has to adapt when an ambitious idea does not work as planned. Most importantly, we learned that AR can be more than a visual feature. It can become a new interface for shopping, where anything you discover in the physical world can become something you take home, experience virtually, and purchase digitally.
What's next for TAGS
Our next step is making the virtual shopping experience even more realistic through accurate sizing and fit prediction, allowing users to understand not only how an item looks, but how it will fit their body. We also plan to expand our extension beyond the sites currently supported, and introduce user accounts with persistent digital closets and social sharing. This would allow every item a shopper discovers to become part of a personal, evolving wardrobe that they can revisit, try on, share, and purchase from anywhere.
Log in or sign up for Devpost to join the conversation.