Find an easy way to analyze photos of camera traps to provide informations about deer populations, their locations and movements in a time course. The deer on the photos should be recognized from the cameras automatically and the recognition results should be stored on a central server for further analysis. An advantage of this compared to traditional systems is, that the AI algorithm compresses the information which has to be sent. This drastically reduces cost of bandwith in IoT environments, because only the detected result and not the raw image is sent.
What it does
Time-series data for deer detections of different camera traps will be collected on a central server. With a web-based user interface it is possible to view, analyze and visualize the collected data.
How I built it
We built a data-sending server that emulates the camera trap which has recognized a deer or something else and sent this result to the central server. This data-sender was built with Node.js. The central server is written in Golang and has a MongoDB as data storage. It provides a REST-Api to be accessible from the data-sender and the web-based user interface which polls the data. The webpage is written in TypeScript and React and can visualize the collected data with nearly real-time updates. The user can see the number and type of deer, the timestamp and its geolocation.
Challenges I ran into
We tried to train a AI model for image recognition (with about 60,000 real photos) to build a prediction engine which should be deployed on the Rasperry-Pi based camera traps. We tried to train a model with a ML.NET based approach from Microsoft with Dotnet core and afterwards with a Tensorflow/Keras algorithm in Python. But both trainings failed due to huge amount of data and limited time. The ML algorithm can't handle the huge size of the pictures (4000x3000 pixels). Unfortunately there wasn't enough time to fully train the Tensorflow/Keras models. So that we focused on the MWP.
Accomplishments that I'm proud of
Connecting a IoT device in the cloud to a centralized analysis server which provides its informations to a webview for data visualization for the enduser. Handling with two AI technologies and earned experiences about their issues especially with huge pictures.
What I learned
Implementing AI model trainings with .NET Core, a REST-Service with Golang and a MongoDB.
What's next for Deer Prediction
Next steps would be to get the AI prediction and model training working. This would also replace the data-sending server that was used here. When the AI engine is working, we want to implement the data transmission with IoT middleware like LoRaWAN or Sigfox. Furthermore we might replace the backend with an open-source IoT solution to facilitate the end user with more capabilities.