TACO Dataset Ease of Augmentation

The TACO dataset is a robust collection of litter and trash images used to train image recognition models, however the process of adding/annotating the dataset is high, so I aim to ease that process.

Comment

Inspiration

Was always inspired to use robotics to solve the litter issue, however I never found an opportunity to step into the space but saw this as the perfect opportunity.

What it does

Takes images from your device and adds them to the localized dataset according to the correct classification with added predictions by the current model. After the images have been classified, retrains the model so that it is more often correct.

How we built it

We used a portion of the TACO dataset (http://tacodataset.org) that had been annotated and summarized it heavily so that the classifcation remained. Downloads the dataset onto your device into proper subdirectories and then trains an initial model. Training is a very basic version of what is linked here: https://www.tensorflow.org/tutorials/images/classification. Afterwards, can use Tkinter dialog boxes to add images from your device, classify them and then retrain the model, saving the complete updated version on the device.

Challenges we ran into

Adapting the dataset so that it was simple enough to be easily added to meant that manual review of multiple entries was necessary to properly classify each image. Additionally images that had multiple classes of litter caused issues so worked to mitigate how the model could predict the most prominent.

Accomplishments that we're proud of

Solving a multitude of bugs regarding tensorflow shape size mismatching and properly adapting the dataset to work with less information and in a smaller environment.

What we learned

Issues with using convolutional neural networks to classify images that might have multiple of the object that is trying to be classified. Deeper understanding of convolutional neural networks and the tensorflow/tkinter package suites Knowledge of using os files to save states, making it easier to store information in the long term.

What's next for TACO Dataset Ease of Augmentation

Hopefully making it even easier to add multiple files at once instead of this slower method. Mobile adaptation or application so files don't have to be transferred to the laptop or device in question (which would necessitate using LiteRT to train models) Web Application so that the dataset can be shared and updated Better TensorTraining, doesn't use augmentation methods or other overfitting fixes to save time (for the sake of testing this project)

Built With

Share this project:

Updates