Image Processor Algorithms working along side a Webcam
We felt that with security cameras monitoring areas 24/7, it becomes incredibly difficult for security and law enforcement to find relevant security footage to use in investigations, as there's too much footage to sift through. Therefore, we believed that if we leveraged ClarifAI's Artificial Intelligence and Image Processing API, we could alert the necessary people of abnormalities in the security footage the moment it occurs, thereby speeding up the process and making investigations more efficient.
What it does
Image processing application that sends push notifications via SMS Messages when abnormalities appear on security footage. The specific configuration designed during the WildHacks 2016 Hackathon event (hosted by Northwestern University) detects for violent behavior, such as fist fights and robberies, as well as the presence of weapons, such as knives and firearms.
How we built it
Utilized the ClarifAI API for Image Processing and the Twilio API for sending push notifications (via SMS), as well as the OpenCV Python Library for communicating with the webcam.
We had to "train" the Image Processing AI (ClarifAI) to understand the concepts we needed (violence, weapons) by showing it various images of both said behavior and normal every day life.
Then we had to extract individual frames from the video footage from the webcam using OpenCV and python, and pass it to the ClarifAI Image Processor.
Finally, we applied an algorithm to the concepts and probability data returned by ClarifAI to determine whether the situation across small collections of frames was considered normal or abnormal.
In the event that the situation was abnormal, we would push an SMS notification via Twilio alerting the designated users of the situation, along with a suggestion to manually inspect the security footage.
Challenges we ran into
We had various compatibility issues across different operating systems as well as programming languages/APIs, which was largely due to differences in dependencies for each technology.
Accomplishments that we're proud of
Integrating all the various technologies together into one streamlined automated flow.
What we learned
The importance of proper and detailed documentation, including all of the different dependencies referenced and installed. This applies to both APIs, as we had some issues figuring out how to utilize certain APIs due to unclear documentation, as well as within your own projects as we often hit compilation errors when porting our code from one machine to another.
What's next for InstaGuard
Parameterizing a lot of the code and data to create a more modular application. This would allow for the application to be applied to a variety of different areas and uses, rather than just detecting robberies and violent behavior. Potential uses could range from alerting home owners when visitors have arrived to notifying train operators earlier in advance of any hazards (such as nearby children or students). Improving portability across machines and operating systems, more detailed documentation for easy installations on clean machines, and improvements to scalability is also under consideration.