Cattle Counter Android app

Cattle Counter home screen

Inspiration

We are trying to help the farmer make sure all of his or her cattle were accounted for at the end of the day. Margins are very low for cattle farmers. Using our drone app they would know if a cow had wandered off or got lost in bad weather. Over the past few years we'd done a number of mobile apps for DJI drones so we thought that if we added machine learning to the mobile app we could hopefully things easier for the farmer.

What it does

Drone flies in an automated fashion over a field of cows at a height of 50ft. Using TensorFlow the app counts the cows in real time showing how many are visible in the video feed at any time. At the end of the automated flight it uploads the tile images to a backend server which combines or stitches the images and gives the pilot a total cow count.

How I built it

5000 images of cows were created from a number of drone pilot flights in the US and Europe. The image labeling was done by a third party service. The original version of the app used Tensorflow 1.0 to train the model. The updated version uses Tensorflow 2.0. App uses the DJI drone SDK to control the flight and display the images. The image recognition is done within the Android app using Tensorflow Lite. OpenCV is used on a backend server to merge the tiled images and Tensorflow 2.0 is again used to return a total count.

Challenges I ran into

Accuracy is the number one criteria for this app. In earlier iterations of this app, we were failing to correctly identify all the cows in the video feed. We recreated the images so that they were taken directly above the cows and not at an angle as well as making sure the flights were taken at a constant height of 50ft. The labeling in earlier versions was not done by professional labelers so that also affected the accuracy. We also used Google Cloud Platform to do the training second time around as the training was taking too long on local machines. If the cows move a lot we will double count them in the final total count.

Accomplishments that I'm proud of

TensorFlow 2.0, better labeling and restricting the count to always be straight down meant that the accuracy of our cattle counting jumped. You can see the final result of the app working in the video demo link.

What I learned

Poorly labeled images will make for an inaccurate object detection model. It's ok to restrict the conditions so that the drone flew in the same conditions each time, i.e. camera pointing straight down at a height of 50ft. Flying any lower than 50ft also means that the cows will move a lot more, which will result in a poor total count.

What's next for Cattle Counter Android app

The app will next be available on Parrot Anafi drones. We're looking to add more species such as deer, elk and caribou. Finally we'd also are planning to complete a thermal image version of the app.