Suitability

Curated outfits for a beach vacation
Workflow of integration

Inspiration

Planning appropriate clothing choices for events (hikes on remote mountainsides, snorkeling trips along the Bahama coasts, candle-lit dinners on rooftop restaurants) is more than stressful: it's unreliable due to the unpredictability of a new environment. Using voice interaction and camera recognition, we simulate these virtual environments in Meta's Quest 2 and find the perfect outfit for your occasion. We're the push away from fast fashion: we find pieces you'll love and fall back on, time and time again, to dress for life events in style.

What it does

Our solution has two fronts: an augmented and AI-communicating reality and a recognition & recommendation website. The user talks to their VR set, indicating what occasion they're shopping for and where they'll be. A camera scans them to analyze their styling choices. Then, augmented reality populates: they're transported into their future event environment, and their ideal clothing choices, based on style recognition, voice commands, environment climate, and occasion, are laid out in front of them. They can review suggestions or interact with the VR until they find a suitable choice.

Suitability finds staple, long-term pieces that transition away from fast fashion. We pull from hundreds of brands, so brand and product discoverability of brands increases, bringing profit for businesses. Clothes that remain on clearance racks for months are sold faster now, because we find recommendations tailored to people based on current-existing clothing; clothes now aren't thrown into the junkyard.

How we built it

We use Faster Wisper to translate user speech to text, and sentence transformers to convert into API search commands. These are used to generate the user's environment via Poly Haven HDRIs. Use DeepFashion2 to identify relevant clothing matches. Use camera integration to identify consumer's clothing style, Open Meteo for accurate environment climate data, and finally FastAPI to convert all these data points into recommended outfits. We use Unity to develop the augmented environment the user describes. With the recommendations from FastAPI, we set the outfits in their augmented background for the user to pick their favorite. Then, they can go to the connected website and scroll through lesser percent-match recommendations, edit recommendations, and finally purchase the outfit.

Challenges we ran into

Biggest challenge we ran into was dressing our human-like avatar with clothes that clung to their body in the manner of tailored clothing. This process was highly dependent on multi-physics factors such as BMI index of the avatar, wind/breeze conditions of the environment, body and face shape of the human-recreated avatar, and stylistic preferences. Setting up integration between META Quest 2 and Unity was arduous due to inconsistent linking softwares and incompatibility between these.

Accomplishments that we're proud of

We fully integrated virtual environments into Quest 2 that the user can interact with via voice, and we identify key clothing pieces that suit this environment (stylistically, practicality, aesthetic fit). We developed a website that can translate these augmented interactions, add on camera recognition of the user's style, and fine tune stylistic recommendations to elevate their current choices.

What we learned

We learned that layering clothes, realistically and physically meshing them, onto avatars is a problem that even top research labs have yet to solve. We learned that pivoting mid way is acceptable, and refocused our problem to provide accurate recommendations and a realistic augmented experience as opposed to a technically robust clothing swap.

What's next for Suitability

We're incorporating a virtual mirror: you see yourself, virtually, and can select a particular clothing option and swipe through a database of outfit options until you find your perfect fit. Essentially, we're incorporating our website into our augmented reality to eliminate endless trial room fittings.