Crisp.AI

Inspiration

For our project, we drew inspiration from the food waste epidemic, specifically with regard to produce. Our team figured that with a convenient, handheld solution, the everyday user will waste significantly less produce with a conscious concept of its freshness as well as recipes catered to its remaining lifetime to rot.

What it does

Crisp.AI takes a picture of a fruit, classifies it, then accordingly produces a freshness metric. After this, it produces recipes based on the current freshness of the fruit.

How we built it

The front end was accomplished using the React framework in Typescript and Tailwind CSS for styling the UI. The back end uses Convolutional Neural Networks (CNN) on the Amazon Sagemaker platform to first classify the fruit in the input picture, and then compute a ripeness score for it using existing training datasets.

Challenges we ran into

Our biggest challenge was trying to link the Sagemaker, S3 buckets, and frontend UI. Cloud deployment has a lot of varied documentation and is heavily contingent on the machine learning models implemented so it's hard to find accurate resources. Currently our demo doesn't work on cloud and therefore the fruit recognition AI doens't work on other devices aside from the local environment device.

Accomplishments that we're proud of

We are proud of effectively creating a classification model with a high average accuracy and our corresponding algorithm to quantify a freshness score. We are also proud of building a functional and clean UI which prioritizes usability and synergy of components over raw aesthetics.

What we learned

We learned the struggles and pitfalls of deploying an image recognition model on the cloud, including the extensive training time, many iterations of restructuring based on false positives/negatives, and how to better use AWS services for the purpose of web app creation.

What's next for Crisp.AI

We believe that while Crisp.AI has a useful implementation for individual clients as we designed it within the scope of this hackathon, the technology can be expanded to track supply chains in produce retailing. It can help stores prioritize what food should be prioritized for the shelves, what should be donated, and what is too far beyond saving. Our hope is that Crisp.AI will soon be able to integrate batch features for scalability and implementation in these fields.