Inspiration
We've all worn shoes well past their expiration date. We've also thrown out shoes way too quickly. The former causes foot pain and injuries to joints surrounding the area. The latter is a waste of money. We do not have the best judgment as to when to replace our shoes. That's why we created The Amazing Shoe Wear and Tear Analysis App of Awesomeness (aka TASWATAAOA)--to determine better when it's time to put the shoes away for good.
What it does
The app allows users to upload pictures of their shoes. An algorithm will determine the quality of the shoes and when the user should replace them.
How we built it
This app was built using a React Native frontend and a Python/Flask-based backend. We trained an LLM using retrieval augmented generation (RAG) on various pictures of shoes to determine the quality of shoes and when it's time to replace them.
Challenges we ran into
Passing the uploaded images to the backend is difficult because of the way images uploaded by the user are handled. The app only gets a link and not the image itself, and it's really tough to extract the contents of the image to pass to the backend for processing.
Accomplishments that we're proud of
This is one of our first projects involving AI, so it's great that we finally get to use it in a practical environment. When used correctly, it is a very powerful tool.
What we learned
We learned how large language models work and are trained. We also learned various means of training models (RAG, XGBoost) and architectures (VGG).
What's next for Shoe Wear and Tear Analysis
The next thing to do is to make this more robust. Currently it only determines whether the shoes are good or not good, but the quality of shoes is a spectrum, so the app should be able to support that.
Log in or sign up for Devpost to join the conversation.