Inspirations

We wanted to create a image recognition scheme that could be used to distinguish food that is safe to eat and food that is unripe or overripe. Using this scheme on bananas, we’d be able direct ripening bananas to store shelves while browned bananas could be used by bakeries to create boppin’ banana bread.

By tweaking a few thresholds, we could easily distinguish other foods like strawberries and avocados. This scheme could also be useful in laboratory settings for parsing images of lab samples that change color or emit bioluminescence.

What It Does

Feed a picture of bananas into Appeal, and it will tell you how ripe the bananas are! Bananas are classified as unripe, ripe, or overripe. The site also provides some interesting factoids about each group of bananas.

How We Built It

Appeal is built using HTML/CSS, Javascript, React and a LOT of Python modules, including Flask, NumPy, and OpenCV. We also leverage the Google Cloud Vision API to identify bananas and locate bounding boxes for them.

For each found banana (or bunch of bananas), we clear the background of their bounding box and feed it into an algorithm that counts RGB values and determines ripeness. Then we determine ripeness for the entire image through a simple majority ruling. This is all done with the help of Python.

The Appeal site is built to allow users to quickly and easily upload images and analyze them. Upon clicking a button, the site calls the Python function that manages the backend computations and analysis. This function returns one of three values, which the site then relays as unripe, ripe, or overripe to the user. This gives immediate feedback to the user.

Challenges

  • Reading Google’s documentation and learning how to use their Cloud Vision API
  • Learning which Python image modules were best for our purposes and how to use them
  • Finding threshold values for contour functions that maximized accuracy for differentiating the banana from the background
  • Determining how to efficiently calculate a sum of RGB values and determine ripeness based off that sum
  • Learning how to transfer the user’s image from frontend to backend
  • General communication between frontend and backend
  • Uploading an image to a server
  • CSS struggles

Accomplishments

  • Learned how to use Google’s Cloud Vision API
  • Learned how to extract values from a JSON object using Python and account for missing values
  • Learned how to use Python’s OpenCV
  • Successfully processed RGB values without leveraging machine learning models
  • Achieved communication between frontend and backend

Built With

Share this project:

Updates