Inspiration
Traditional e-commerce search relies heavily on text. Yet, human thought is often visual, intuitive, and difficult to articulate in words. We were inspired by the gap between how people naturally imagine products and how they are forced to search for them online.
What it does
Shop-A-Sketch lets users draw what they want to buy, then uses AI to interpret their sketch and search real products via Google Shopping APIs. Results show prices, ratings, and store links, and users can filter by price to refine their search. Users may then click directly into the item to purchase from the merchant.
How we built it
Our drawing canvas is built with React and Tailwind for a clean, responsive UI that works on any device. Sketch input is processed through Gemini and Python-based services to extract semantic meaning from rough drawings. That understanding flows into our identity layer, which modulates results based on user-declared values. We query live product data through Google Shopping APIs and return results in real time. Everything is containerized with Docker for scalability, and our design was iterated through Figma to ensure the experience is polished and accessible.
Challenges we ran into
Finding an API to search for products on the web.
Accomplishments that we're proud of
We built a seamless flow from sketch to product in seconds, powered by a multimodal AI pipeline that interprets rough drawings. The result is a clean, responsive UI, making visual search intuitive for everyone.
What we learned
Tailwind!
What's next for Shop A Sketch
Looking ahead, Shop A Sketch has the potential to revolutionize how users discover products, particularly in large-scale platforms. By enabling a more intuitive, visual-based search experience, we can shift the focus from traditional text-heavy search to something that aligns more closely with how people naturally think and visualize.
For a platform with a large selection of products, this could significantly reduce the barriers to finding the right items, especially for users who struggle with specific keywords or product descriptions. As the AI model refines its ability to interpret sketches with increasing accuracy, it could also unlock personalized product recommendations based on individual drawing styles or preferences, improving overall customer satisfaction.
The ability to process large volumes of user interactions in real time, combined with integration with product catalogs, could transform product discovery, offering a more engaging experience.
On the backend, scaling the service to support millions of users with diverse search patterns would require robust infrastructure, focusing on high availability and AI model optimizations. As more users generate and refine their sketches, the system could continuously learn and adapt, improving its ability to provide accurate results with each interaction.
By leveraging these advances, we could open the door to a completely new way of shopping, where product search becomes more interactive and personal, catering to the evolving needs of a global, tech-savvy consumer base.
Log in or sign up for Devpost to join the conversation.