Realtime Updating Webpage for Alerts
Demonstrating the Application with a Phone (Ideally this App would be coupled with a Drone)
Sample Data Submitted From a Mobile Phone Flowing Along the Data Pipeline
We wanted to build a harmful weed detector for the use in the agriculture industry. However, data was difficult to find. We were very attracted to the idea of object detection so we decided to apply computer vision on livestock.
What it does
An endpoint device, such as a mobile phone or a drone with a camera, sends an image to the backend. The backend automagically outputs the location and image of sick animals. This lets users know where and when to help animals.
How we built it
We gathered many images of the eight most commonly found farm animals: cow, pig, horse, sheep, goat, donkey, rabbit, and chicken.
Then we trained two machine learning models, one dedicated to object detection where it draws bounding boxes to isolate the animal from the rest of the image, and another model was used to classify the health of the animal as unhealthy or healthy. (A classifier model was built for each animal.) Both were trained solely on images. We used Tensorflow's Object Detection API and Google Cloud's Vision API to create the best models we could with our dataset.
If an animal was classified as unhealthy, the base64 representation of the image is stored in a Firebase Realtime Database. Results and the alert are sent to the website in realtime without needing to refresh.
The React framework was used to develop the mobile app and the web app.
Challenges we ran into
Creating our own dataset and manually annotating each image was very time-consuming. On top of that, it was difficult to find a significant amount of photos for our unhealthy animal datasets.
Developing for multiple platforms increased the complexity of the system, so there are multiple points of potential failure.
Accomplishments that we're proud of
We made a working product by stitching together many helpful API's and frameworks. The product can be demonstrated successfully.
What we learned
It's very difficult to achieve outstanding results with a limited dataset. Maintaining a system with multiple potential points of failure can be difficult to troubleshoot and debug.
What's next for EIE.IO
Because of its potential extensibility, a model for assessing other important entities such as soil quality, invading crop weeds, and growth monitoring could be trained and added to the pipeline.