MOLD.ai

Inspiration

You know that feeling when you finally examine your fridge after a long week of work and school? Oh how painfully soul-crushing once you see those groceries you promised yourself you'd eat, but let them go to waste...again. Well, with MOLD.ai, you'll receive real-time updates on which foods are on their way to ripe-ville, so you can eat them before they end up in the trash!

What it does

MOLD.ai is a computer vision algorithm that can automatically detect whether or not a food item is turning past its prime ripeness.

How we built it

Using Google Cloud Vision AI API, we were able to take the most prominent colours from the food item. From there we created an algorithm that compared colours based on a variety of factors, specifically looking for the similarities in range of the colours. From there the algorithm spits out whether it thinks it is moldy or not and displays the most prominent colours of the food to the viewer. The display is completely handled by Java's Abstract Window Toolkit

Challenges we ran into

We were initially locked out of using not only Google Cloud, but my Google account entirely. This set a major roadblock as we had to contact Google for support and went through a long and tumultuous process for the safe return of my Google account. We also had difficulty deciding what assumptions our algorithms could make in terms of colour comparison, as the calculations were very precise with hard cutoffs between the two outcomes. Finally, due to these previous unforeseen difficulties, we were limited in the number of hours of sleep that were available to us, which led to less productive working hours and somewhat confusing code at times.

Accomplishments that we're proud of

The thing we are most proud of is properly implementing the Google Cloud API into our project, as we were initially very overwhelmed by the sheer capability of the API. We overcame our confusion and became mini-(mediocre) experts with the API. We also did not have much previous experience working with an API of this size let alone any AI software. Finally, we were able to complete this project with only two members!

What we learned

We learned to never assume anything, especially when it comes to installing or learning new software. We COMPLETELY underestimated the steep learning curve of working with this API. We also learned that we don't really write the best code when we are very tired. Finally, we learned that with enough dedication, coffee, tea, and goldfish, you can make anything possible, even when Google tries to suspend your ENTIRE PERSONAL GOOGLE ACCOUNT (:

What's next for MOLD.ai

We initially meant to completely redesign the fridges of today, by adding scales and cameras/sensors in the interior to support the typical college student in their meal plan endeavours. The scales would weigh food items and sensors would be collecting data on the changes to foods like fruits and vegetables that may mold over time. This data would be transformed as not only updates to the users that their food is at its prime ripeness (or a bit past), but also suggests recipes based on their food inventory (given by the scales). We would like to dive deeper into the API realm so that those meal recommendations would be provided by the most relevant search results, thus, centralizing all things food onto a single program.

Built With

• google-cloud
• intellij-idea
• java
• java-awt
• maven
• vision-api