Project SafeSight

Landing page
Adding your phone to phone list
Dashboard
Heatmap of Incidents

Inspiration

The recent upward trend of gun violence plaguing America and our local community should come to an end. We've noticed many news outlets have CCTV sources that capture such events however first responders are only contacted about threats after or while the damage is being done. What if innocent lives and police are notified before such events take place?

What it does

We analyze footage real time to alert of such threats as soon as they are seen in frame. (For the purpose of this hackathon we tested upon frames of CCTV footage from news and other sources). We've expanded upon this core feature to introduce heat map visuals, and notification distribution (between private groups [ex. businesses], and general geographic regions), frame/snippet previews, and more.

How we built it

Tech Stack

NodeJS
ExpressJS
ReactJS (in Typescript)
Azure
MongoDB
JWTs
Google Maps API
OpenCV
Python
Twilio

Using Azure Machine Learning, we managed to train several iterations of our weapon detection model with a dataset of approximately 900 labels. The Azure platform includes the ability to use these models in real time through an API. Using Python and OpenCV we tunnel every Nth (configurable [for the sake of saving bandwidth]) frame from a VideoCapture source to the Azure API which scans for weapons. In the case a weapon is detected, a bounding box and confidence score is illustrated upon the raw frame and uploaded to a cloud service. This new image will be included in alerts sent to a database of phone numbers with Twilio.

Data including phone numbers, logins (hashed), previous events, and more are all stored within our MongoDB Atlas database. Using an ExpressJS app, we serve read/write requests called from the ReactJS web app.

Challenges we ran into

We wanted to create a makeshift surveillance camera with a raspberry pi and Google Coral USB Accelerator. This would run a TFLite model. However, the exported models from Azure are "condensed" and did not perform up our standards.
Spotty Wi-Fi here and there
Internal issues (different visions of the end product)
Issues training the model during some iterations
Picking a project name 😭

Accomplishments that we're proud of

Using the Azure Machine Learning platform (first for all of us)
Completing the project (had doubts at the start of Bit Camp)

What we learned

We all contributed to this project in accordance to our strengths so we all shared knowledge with each other, curating an environment for growth for one another.

What's next for Project SafeSight

Camera's can only see so much. Introducing an audio aspect to recognize fire is something of interest.
Stress testing the project infrastructure (beyond hackathon conditions)