AI’s potential impact in agriculture is HUGE, transforming the landscape of sustainability forever. However, this potential will remain just that without access to high quality datasets.
Currently, teams are spending up to 80% of project time obtaining and cleaning image and/or video data sets before they even build AI for agriculture. This is because companies must go through stringent privacy regulations and manually clean data with human input which is a manual, error-ridden not-to-mention biased process. Plus, of the few available, current available datasets lack diversity. This translates to companies spending 3-5 years to build performant AI algorithms.
Our team has had first hand experience with these issues in machine learning having faced them in large research institutes and corporations. We thought there had to be a better way. So we decided to flip the paradigm with DataSynth to solve the data bottleneck in vision AI for agriculture.
What it does
With the click of a button, DataSynth generates infinite amounts of automatically structured (labelled) image datasets that machine learning teams use to train AI algorithms for agriculture. A user simply inputs features they want in their dataset and, after pressing submit, they have it readily accessible via a Google Drive folder. We produce both the raw RGB image as well as the corresponding annotation file that are required to train ML models. An example could be if I want 10 pigs and 100 images in a farm.
These images do not lack diversity either. Using state-of-the-art game design techniques such as procedural generation, we are able to place objects and terrains randomly in scenes to imitate the variation of the real world. This produces richer datasets than what can be usually found in real life!
As a result, we expect companies and ML practitioners to be able to build vision systems in agriculture an order of magnitude faster at a significantly reduced cost.
How we built it
There are two key components: the web app and image generation. Firstly our Flask web app deals with user authentication as well as taking in inputs from the user. This connects to Unity via their API to generate data. We have designed a scene of a farm within which the field is randomly generated with procedurally placed assets that the user inputs, like the number of pigs and hay barrels. Cameras and lighting also automatically vary. Additionally, our build runs a simulation of “drone” footage where we perceive and label the ground objects. We then are able to generate labels as well as raw images using the Unity Simulation packages which are able to be interpreted by a machine learning model. These generated files are sent to Google Drive via their API and are accessible via our dashboard built on Flask as well.
Challenges we ran into
Nothing :) Ha, jk.
We ran into a healthy handful of challenges, as one does at a Hackathon. What’s important is that they didn’t stop us.
- Deploying a flask web app is deceivingly difficult.
- Managing merge requests
- Dealing with the Unity Simulation API and in general was quite difficult due to the lack of documentation available.
Accomplishments that we're proud of
- Got the Unity and Google APIs working with very ~unhelpful~ documentation.
- We were very well organized throughout the entire Hackathon, communication is key and we really nailed it.
- A unique and really polished final product.
What we learned
- Flask is great.
- Bad documentation really just isn’t a vibe or conducive to effective use of an application.
- Doing a Kahoot with Katz has a lot of American Politics involved.
What's next for DataSynth
We currently have modelled one environment within an agricultural setting. Given there are numerous use cases of vision AI in agriculture, we would like to simulate various other scenes to generate data. For example, weeds within crops.