Agriculture is one of man’s oldest and most vital industries. Since humans have grown crops, we have faced the problem of disruptive insects that reduce crop yields. Farmers began using pesticides in the early 20th century to combat these pests. While effective, overuse of pesticides wastes money, hurts the environment, and necessitates further pesticide use as the pests mutate. Modern farmers try to minimize pesticide use whilst maximizing yields. This art/science is called Integrated Pest Management (IPM).

The central problem of IPM is determining how many pests are present and thus how much pesticide is needed. The current method is to place sticky tapes at regular intervals in the crop field and visually inspect them every couple weeks to see how many dead bugs are on the tape. Based on the findings, the farmer will often hire an IPM consultant to determine the proper course of action.

In this project we develop an IoT device and expert system to replace the tapes and the IPM consultant. The device consists of a modified tape, raspberry pi, and camera which relays raw images to the cloud for ex-situ image processing and analysis.

How To Use

1) Connect raspberry pi to the internet. Install OpenCV library.

2) Clone our repository (it's private, email jbecke at seas dot upenn dot edu for access). Change the permissions of the private key by running "chmod 400 *.pem".

3) Your can run the python script yourself by taking an image and running the python script (cd into the git directory first) or you can add our crontab script to your system's crontab job file to have it run automatically at regular intervals

4) Place the raspberry pi and camera approximately two feet away from the sticky tape you wish to monitor.

5) When pictures are taken they will be automatically analyzed to determine the number of bugs and the data will be uploaded to the cloud! View the data and raw images at ec2-54-144-59-153.compute-1.amazonaws.com

Technical Description

See the video for a technical demonstration. Idam Prezi

A python script called blob.py is run on the raspberry pi. It can be run manually in terminal or by a crontab job. The script has three functions: (1) taking an image, (2) performing blob detection and counting on the image, and (3) ssh-ing into the AWS EC2 instance and running "bash adder.sh [pest count]". The adder.sh script (in the ~ directory of the EC2 instance) performs a simple SQL insert query on our testFarm table in the pestdata MySQL database. The front end consists of a PHP file, images, and the FusionCharts libs and is located in /var/www/html.

We decided to use a simple EC2 t2-micro instance with the back-end running on the same instance (apache, localhost) instead of connecting to RDS because we didn't need the load balancing and extra features that come with that product. If we were to expand and commercialize our product, this would be the proper way to do it.

More info on:

(1) The image is taken using the PiCam's python library and named image.jpg. If you wish to supply an external image you can name it image.jpg and put it in the same directory as the blob.py file.

(2) The blob detection is done using OpenCV, and open-source computer vision library written in C++ (we use the python wrapper). First, the image is converted into a bitmap and a median blur with radius 15 is used to make the insects more blob-like. Secondly, the image is thresholded to get dark pixels (because we assume our bugs are darker than the white or yellow sticky tape). Then OpenCV looks for closed paths created by contours in the image. If a cycle is found, it then checks to see whether the cycle satisfies the defined parameters (you can play with these in the file) such as convexity, circularity, area, and color bounds. If it does, it gets the coordinates of the blob's center and appends it to the array of blobs. At the end of this process, we count the length of the array and save it as "int numBlobs".

(3) The script uses python's os.system(str cmd) function to run a bash command on the local machine. In our case, the command is to ssh into the server using the private key, run the adder.sh file with the current date and number of bugs, and then exit the ssh tunnel. Next, os.system is called again to scp the image over to the server's /var/www/html/images folder and name it the current date and time.

Built With

Share this project: