tailor maid

Ever been conflicted in the morning as to what to wear? Yet, if you pick a random outfit, it is either too warm or too thin? Fret not, for we now have a solution, tailor-made for you!

Comment

Inspiration

One of my classmates almost arrived late to the midterm because their wardrobe was too huge and choice paralysis was too strong. For days like those we thought it would be very useful to have an app suggesting a comfy outfit for the day without the need to choose, since choice paralysis is real.

What it does

tailor maid lets you upload pictures of clothing articles which the app will remember and sort, then use AI to display an appropriate outfit based on the weather every morning, or for a specific prompt of your choosing. It is an app that can be accessed from any browser and can also be activated by voice command for ease of use.

How we built it

Our backend is in python and we did all of our development on Google Colaboratory since we use AI extensively. The libraries we used include keras, tensorflow, PIL and numpy. First we use a weather API to get the current weather for our city (Montreal), then we classify uploaded pictures (from any device) according to the imagenet 1k synsets. Then we can generate a list of suitable clothing for the weather and suggest an outfit for the user. This happens very quickly, and generally within 5 seconds you have your outfit for the day! And you'll never feel too hot or too cold in it :D

For the frontend, we used anvil, which has a very neat integration with Google Colab, at the expense of a very finnicky drag and drop design for UI elements. Anvil also automatically creates a Github repo and a link to the website, so it was also trivial to deploy this project. We could then focus most of our efforts in the backend.

Challenges we ran into

The technology we are using was cutting edge, with the imagenet21k being released only months earlier, meaning we have pretty much no documentation to help us debug.

one of our teammates kept playing games

Google's speech API also has outdated elements, so we had to find a workaround by ourselves. We also need to take into account the different limitations of every API we used. For example, converting between PIL, raw bytes, numpy arrays and anvil objects as we pass images from frontend to backend, and for audio we had trouble adjusting for different pcs since the microphones had different settings, etc

Many of our planned features were also scrapped because we lacked the time/resources to implement them, or because the current technology available was unable to meet our requirements. One such example was the ability to add a batch of clothing via amazon links, we looked into it but amazon's API was paywalled and required very complicated f2p workarounds that we wouldn't be able to debug in 36h

Accomplishments that we're proud of

Just getting the pretrained imagenet model to work was a lot of work, and we relied on a Github issue thread authored literally days ago to fix our problem. This really gave us a taste of what it's like to work with beta software. We initially used Alibaba's implementation of imagenet from 2017, but one of their dependencies straight up just bailed out and never updated ever again, so we decided to look for the model with the most recent updates, which is how we ended up with EfficientNet. Our implementation of imagenet can also recognize other objects and their colors, but for this project they are not relevant and we did not optimize it for that.

We are also extremely proud of being able to pass audio from the browser to the Colab backend, as it took a lot of workarounds for free APIs and deprecated libraries

What we learned

AI changes so fast that documentation is really hard to track. Also learned a lot about the different APIs we used and resolving common issues encountered while using them.

We learned a lot about how to chain different API calls together to make them work as a single application.

What's next for tailor maid

Add integration with google nest/other voice assistants move the project to a normal python backend after we finish developing on Colab train our own model to be better at recognizing clothing finish adding imagenet21k with improved thickness detection improve speed

Built With

Share this project:

Updates