Inspiration

Getting ready in the morning is hard, a mirror can show you what you are wearing, but sometimes there is just something that the outfit is missing. We wanted to make getting feedback feel natural and easy. Plenty of people post their fit checks on instagram and tiktok, so we used that format to give people comments about what they are wearing. The audience is simulated, and the feedback comes from what the AI sees during your session.

What it does

Turn on your camera, step back, and show off your fit. The 15-second runway starts once oko recognizes clothing. As you turn or bring a detail closer to the camera, comments appear with specific compliments and practical styling suggestions. After the runway, oko guides you through three poses and takes each photo automatically. Your personal looks feed brings together the photos, comments, and an outfit score with a breakdown you can expand.

How we built it

We built the React interface and server API routes in one Next.js app. The browser records silent 2.5-second clips and sends Gemini a rolling window of up to five seconds. Gemini identifies visible pieces and ties each comment to a garment it observed. The app remembers pieces across clips, gives newly revealed items a turn, and spaces comments out in the chat.

For the final review, Gemini looks at up to three images from the session and supplies an overall score plus estimates about the visible garments. Our rules use those estimates to score how the clothes fit and drape, which piece draws the eye, and how the outfit fits together. The final score averages Gemini’s overall score and the rules score equally, and the feed shows both. Photos and full reports stay in the browser through IndexedDB. Firebase Anonymous Authentication and Firestore handle private storage for score records and selected comments.

Challenges we ran into

The hardest part was making the chat feel live when the model takes time to respond. We limited each session to two live analysis attempts and one final review, then queued and paced the replies in the browser. We also found that WebM decoding was unreliable in our tests, so we used MP4 where supported and added a fallback that sends a short sequence of timestamped frames. We also ran into issues with API rate limits and had to balance responsiveness with API costs.

Framing was another challenge. A single frame might show a jacket but miss the shoes, and someone turning around can briefly leave the frame. Looking across clips and remembering which pieces we’d already seen let the runway keep moving through partial views while cutting down on repetitive comments.

What we learned

A specific suggestion like “try rolling the cuffs once” can be more useful than an overall rating. Getting to that suggestion took more than writing a prompt. We had to give the model enough visual context to see the cuffs, check that the comment referred to a piece it had actually observed, and deliver it while it still felt relevant.

Try oko · View the source on GitHub

Built With

Share this project:

Updates

Submission history