Saved Items (Right now only one)
Signing up for an account
Posting the picture
Online results after clicking the shopping bag. The actual shirt couldn't be found, so it found the best match.
Feed with Posts (I Liked this one!)
Firebase Storage for Images
Firebase Database for Data
Putting the Two Dots on for Augmentation
For us, most things we buy are online. Toys, electronics, even food sometimes. But for clothes, that's almost impossible to do. Website after website outline the downfalls that come with online shopping; mainly, not being able to try things on and see how they look, and not knowing what to get. Several people we talked too said that the only reason they don't shop online are for these reasons. Others said this combined with their laziness to drive to the store meant that they decided to not buy the product at all. We wanted to create a social media for fashion, allowing users to benefit all the rewards of shopping from the comfort of their home. This not only makes it easier for users, but also increases the number of actual purchases from stores using this app.
What it does
When first logging in, users are prompted to either make a new account or log in as an existing user. Next, you are directed into the main feed. Here, you can see posts from other users and like, comment, save, and try on their outfits. Clicking on the middle-bottom brings you to all your saved outfits, from where you can try them on or buy them online, which will redirect you to the online retailer for that piece of clothing. Trying on an outfit consists of tapping your right shoulder and your left hip, and then having that outfit augmented onto you in a picture. Finally, clicking on the bottom-right brings you to the page for posting a picture. There, you need to submit a post picture, a picture of the piece of clothing so we can augment it, a caption, and a tag of where the item can be found online.
How we built it
All of our online storage, for posts, users, likes, captions, links, and profile pictures, is done through our Firebase server. We used the real time database for simple data like numbers and strings, and the cloud storage for our pictures. We used Apple's swift for all the UI and the interfacing between screens. Lastly, we used OpenCV, a computer vision API library, for all the extracting and augmenting of photos, along with taking new photos. For the extraction, we use bit-masks for color filtration of the clothing along with deep-learning to recognize where the clothes are to cut out a precise image of the outfit. We use the two points on an image to know where to put the images and overlay the clothes onto the person.
Challenges we ran into
While using and implementing Firebase was difficult, eventually the documentation helped us overcome those challenges. Extraction of the piece of clothing was another big challenge, but we eventually used deep-learning and bit-masks to solve our problem. Definitely the biggest challenge we ran into was augmenting the photo onto the human. We tried to do this without any user input, but we found out there were no APIs that did that, unless we used a Microsoft Kinect, which we thought would be unrealistic for a real person. At the end of the day, we had the user input two points of data and were able to augment the image from there.
Accomplishments that we're proud of
Just making a social media app, regardless of the fashion, was an accomplishment we were proud of, as getting all the UI, online storage, and functionality into one seamless app is quite challenging. We also used deep-learning, an up-and-coming trend in the programming industry, for the first time, an important and useful skill we'll need later. Finally, augmenting the outfit onto the person accurately was rewarding as it showed a promising final result.
What we learned
We learned a ton from this hackathon. Firstly, we learned to read all documentation and not take shortcuts, as that just leads to more wasted time looking at errors down the line. Secondly, we learned lots of specific programming techniques, like deep-learning and online databases. Lastly, we gained experience. Because two of our three members had never been to a hackathon before, gaining experience will play a vital role in helping shape the rest of our lives.
What's next for WearIt
There are a couple things we want to do in the future. First, we want to have the outfit be augmented on without any user input. Secondly, we want to be able to extract the outfit from the post itself instead of taking a seperate picture for that. Thirdly, we want to be able to try on more than just shirts. Finally, and most importantly, we want to make the augmenting more true to real-life, as right now it doesn't look real.