Rotten Bananas

Inspiration

The concern for a clean environment implies limiting food waste, we though of automating food waste detection in a warehouse environment.

What it does

Classify bananas seen though a camera as fresh or rotten.

How we built it

We build a model based on Yolov5 with PyTorch and we trained it with a dataset of images of fresh and rotten bananas that we collected. We used Google Colab to train our model using the GPUs they provide with a free account. We then built a python server which fetches the live video stream from a Raspberry Pi's camera, passes it through our neural network and sends the result via websockets to a web client written in javascript. That web client then displays the result.

Challenges we ran into

• We did not have time to build a proper data set to train our model.
• We did not have an actual rotten banana to test our system at the end.
• We tried to do the inference step on the edge at first using either ONNX or TensorFlow.js, but we encountered compatibility problems that we did not find a way to fix, so we decided to do the inference step on the cloud instead.
• The system had a very low frame rate (often less than one frame per second), this is because we ran the inference on a node which did not have an Nvidia GPU, hence CUDA was not available and we ran it on the CPU instead.

Accomplishments that we're proud of

• Getting the model to work decently in the time we had without having a good dataset for it nor enough time to make one.

What we learned

• How to train Yolov5 with a custom dataset
• How to use websocket with python and js
• How to serialize and deserialize a video

What's next for Rotten Bananas

• Extend to other fruits
• Integrate a routing system to send the fruits in the right place depending on their class
• Integrate a map widget to display food waste collection centers

Built With

• javascript
• opencv
• python
• pytorch
• websockets