Our generation is obsessed with online shopping, especially during quarantine. I personally have online shopped a lot, and recently, I have researched about the damage this is causing to our planet and the health of thousands. The textile industry is one of the most polluting industries in the world , and due to fast fashion brands like Zara and Shein , buying cheap, trendy clothing is appealing. However, the waste from making these items is damaging waterways, using up hundreds of gallons of oil and water, releasing more emissions than flights, and the chemical runoff is leaving people with medical health issues . We can’t just tell people to stop shopping. Instead we can teach people how to become smarter shoppers and help them save money and help the planet at the same time. So, I have developed Shopala , the first machine learning shopping assistant app of its kind.
What it does
It calculates cost per wear of items that users want to buy while they are shopping, based on items the user already has . When users take a picture of items they want, the machine learning used enables the app to compare the image to previously imputed images of clothing that the user already has. It takes the cost of the new item and divides it by the number of times users wore a similar item in their closet.
How I built it
I used react native to develop the mobile application. I used the React APIs to render the components on a mobile device. I also used Clarifai API to incorporate machine learning. I created a custom model to get the model to begin understanding differences among specifically clothing and store json metadata of the amount of times users wore that particular item. As users input more apps, the model becomes more intelligent in recognizing differences in apparel, fabric, and clothing types.
Challenges I ran into
Getting the imputed information to enter the model and be set as json metadata for each particular image was difficult. Fetching this data when users selected an image similar to it was also a challenge. However, I read a lot into json metadata and its functionality, and learned a lot considering I have never heard of this functionality prior to this hackathon. After a lot of trial and error, I was able to narrow down the hits of similar images to 10, and have the metadata return. After this point, it was just a simple arithmetic function that provides the user the cost-per-wear value.
Accomplishments that I'm proud of
Creating a whole custom model and getting the json metadata to work was something that I was surprised to do in just 24 hours. I have some experience with machine learning, but I have definitely refined my skills a lot, and learned about connecting user input into a machine learning model. I also spent a lot more time on UI and it resulted in a professional looking application.
What I learned
I learned about more React API features in React Native and implemented them for a better UI. I also learned about json metadata and connecting it with Clarifai custom model API. I was able to use this cutting edge machine learning technology to make a mobile application, and develop it to successfully and accurately predict images.
What's next for Shopala
I was pleased that the machine learning aspect works and is accurate, but I would like to continue to develop this app and put it in the app store soon! I want to create a user profile page where users can input their name and set some preferences. By doing this, I can welcome the users on the homepage when they enter with a “hi !”. It’s a small touch, but it makes the world of difference because it makes the app more friendly. After all, it is a shopping buddy. In addition, I would like to include an image gallery of all the previously imputed images and users will have the ability to remove images, or edit the json metadata of how many times they wear the item in a year. Finally, I would like to improve the UI even further. I have gotten amazing advice on moving forward from Vicky Vo, project designer mentor during the hackathon, and have begun to implement some of the features she suggested, but I will continue to edit it. All in all, I am proud that the machine learning worked excellently, and I am excited to continue to develop it following this hackathon.