With Covid-19, the national parks have hit record highs for attendance. However, many people don't know how to identify poison ivy or poison oak. That's where Poison Detector comes in: anyone can take a picture of a plant, and our AI will detect any poison ivy or oak present.
What it does
When the user takes a picture, our AI is able to detect whether or not poison ivy, oak, or sumac is in the photo, and warns them accordingly.
How we built it
Dataset: Images of poison ivy, poison oak, commonly mistaken plants such as virginia creepers and blackberry vines, as well as a few curveballs such as irises and pampas grasses. Data Preprocessing: Our code categorizes and labels our images with "not," "ivy," or "oak" so the model can learn. We artificially created additional data by rotating our images and used OpenCV's grabCut to pick out the leaves from the background. We then send the leaves through a Histogram of Oriented Gradients (HOG) to find the shapes of the leaves. Model: We used a Keras Sequential model with four Dense, neural net layers, and a Dropout layer to prevent overfitting. In the hidden layers, we used the ReLU activation function, and for the output layer, we used Softmax, which makes the output the percent likelihood that the image is poison ivy, poison oak, or neither, respectively.
We trained our model with 100 epochs on over 300 images, with 41% accuracy on the training set. The model was able to achieve 35% accuracy on the test set (comprising 20% of the data as per the 80/20 rule) and 67% on our very small evaluation set.
Challenges we ran into
We were unable to find a dataset of poison ivy pictures, so we had to manually create our own. This created its own problem, however, in that the leaves did not have a standard position in each frame, making it harder to train our model.
What we learned
One of our hackers had more experience with artificial intelligence, while the other had more general Python knowledge. This hackathon was a learning curve for us, as we were both new to computer vision and object detection.