This was built as part of my Final Year Project with UCL and Microsoft in which tools were developed to make developing anti-poaching systems easier to build.
What it does
It takes videos stored in Azure storage and sends them to Video Indexer to be analysed. It then extracts the labels from the response and filters out the labels with a confidence value below a user set threshold. These labels are then returned by the script. This is a proof of concept tool that could be used with Azure Functions to automatically send videos uploaded to Azure Storage to Video Indexer to get a quick insight on the videos. In our case, it could have been used to quickly analyse drone surveillance footage.
The ML models are built to detect elephants. One model detects them from a ground view and could be used in camera traps, the other detects them from an aerial view and could be used on UAVs. Both are object detection models so can not only identify if an elephant is in the frame but also how many there are and where they are located in the frame.
How we built it
Made use of the Video Indexer API and was written in python using the requests library to make calls to the API.
ML models were made using Azure Custom Vision.
Challenges we ran into
Could not upload custom ML models for use by Video Indexer, had to rely on the default analysis services. Also no official python wrapper module for the API.
Was not able to use a UAV to collect aerial elephant images as planned so had to make do with public datasets and images I could find online.
Accomplishments that we're proud of
Managed to create a tool that can provide quick insights on a video located in Azure Storage.
Given the small and low-quality datasets that had to be used, the performance of the models was actually quite good.
What we learned
Learnt more about how to use cloud services and how to utilise them within a project of my own. Learnt how to use the Custom Vision Services.
What's next for Video Indexer Script and ML models
Use Azure Functions to develop a system where uploads to a storage account are automatically detected and the script is run to analyse the recently uploaded video.
Collect a better dataset to create more accurate models.