Inspiration
You've probably been there, scrolling endlessly through an online glasses catalogue, only to be overwhelmed by choice, only to end up buying something that doesn't suit your face. Most eyewear stores push the most expensive designer brands rather than help you find something that fits. We wanted to fix that.
What it does
FrameAI lets you describe what you're looking for, upload a selfie, and receive personalised frame recommendations from a range of online eyewear stores, complete with a virtual try-on so you can see how they look before you buy. By using machine learning to find the most flattering frames, FrameAI saves you time and money spent on glasses that you'll never wear.
How we built it
The system is powered by five AI components working together to deliver recommendations in seconds.
- Attractiveness CNN - An EfficientNet-B3 backbone with dual spatial and channel attention learns to focus on the facial regions most relevant to eyewear. Features are pooled into a 3072-dim vector and scored 1–10.
- Frame-Style Matching Network - A shared EfficientNet-B0 backbone feeds four heads in parallel: face shape classification (5 classes), style classification (8 classes), a compatibility scorer that blends a static optometry matrix with learned embeddings (60/40), and a direct attractiveness head.
- CLIP Embeddings + Rocchio Feedback - OpenAI CLIP maps products and text queries into a shared 512–768-dim space. Users refine results via like/dislike, which iteratively adjusts the query vector. Conversational Preference Engine — ElevenLabs handles the voice interaction; Claude extracts structured preferences (shape, colour, style) from the dialogue to seed the CLIP search.
- Virtual Try-On - Google Gemini 2.5 Flash overlays recommended frames onto the user's face for instant visual preview.
Challenges we ran into
We had a slow start. We started off with a Skyscanner price prediction concept, immortalised in our GitHub description, only to find out that Expedia automatically provides this feature. Learning Deep Learning and various other themes from scratch in 12 hours whilst simultaneously building a working product was also a major challenge in itself.
Accomplishments that we're proud of
We've tested our CNN on our team members, wearing a range of flattering to unflattering glasses. Our neural network accurately predicted and suggested flattering frames that matched our preferences, a genuinely exciting moment.
From a rough start and a pivot mid-hackathon, we managed to ship something that we're proud of in 12 hours.
What we learned
Slowing down to think carefully about the idea before diving into code made a huge difference. Had it not been for that, we would have been stuck with a already tried-and-tested idea. With GenAI tools, we've learnt that idea generation is equally, if not more important than diving into the technical details.
What's next for FrameAI
Our vision goes beyond just glasses. We want to scale FrameAI to other clothing and accessories, partnering with brands to become the go-to destination for personalised online retail. Our goal is to put confidence first by not charging users but rather charging brands for exposure.
Log in or sign up for Devpost to join the conversation.