CCTV footage plays a key role in crime investigation. Based on many studies, the very presence of camera surveillance systems has discouraged criminals, thus preventing crimes from happening. It creates valuable evidence for court trials. CCTV cameras are as important as forensic evidence like DNA samples and fingerprints in the investigations. The footage can be used to establish the timeline of an incident or a crime. A 2009 report showed that of the 90 murder cases recorded during the year, CCTV footage was used in 86 of the cases. Therefore CCTV footage is valuable in more than 90% cases!

What it does

We have designed a system to capture the face, expression, and gesture of targeted persons (criminals) through distributed CCTV systems and maintaining it in a database along with time and location stamp. The database-so compiled to be used to identify suspects from video clips of crime-related CCTV footage captured series of CCTV Systems located en-route and close to the scene of the crime.

How we built it

Our project essentially is based on a deep learning pipeline with two different layers. The first layer comprises a face recognition algorithm that can learn and recognize a person's face, in our case criminals face by understanding the patterns that make up the unique features of his/her face. For face recognition to work accurately, we will need a lot of training data, assuming that this amount of data will not be available, we decided to search for this data. So once the face recognition layer is trained on the criminal database, we use its recognition ability to find out images on social media of that person, so that we can replenish our data and update the model to a more accurate version. Once the above training process is complete, Samaritan will be then ready to take in live CCTV footage and put its face recognition muscles to test track, and apprehending criminals or persons of interest, making the streets of this world a safer place, one True Positive Recognition at a time.

Challenges we ran into

Two of the key challenges that we faced are:

Ethical issues : People fear face recognition technologies because it is a threat to their privacy and we are totally for all people. Thus we took certain serious measures, so that Samaritan will never compromise someone's privacy. A few of these measures are like, giving more importance to low false positive rates so that an innocent person will never be tagged as a criminal and not storing the images of people who don't match with our database, and finally relying solely on government's criminal database alone for the core facial heuristics and no other source, so that only criminals comprise our dataset. We are totally to the people, for the people and by the people.

Lack of Data: Assuming that the criminal facial data will be very low and the cctv cam footage will be of standard definition, we set the thresholds real low, so that we can motivate ourselves to build a model that is robust enough to deal with any situation. To replenish our data, we use the criminal facial data trained dl model to find out similar images on social media sites and replenish our datasets.

Accomplishments that we're proud of

The thought that we as a team conceived the idea of Samaritan in itself makes us very proud, as we can now help the government in guarding our cities and our homes in a more faster and efficient manner. We solved a lot of technical difficulties, overcame ethical hurdles and had a ton of fun while building Samaritan. It took a lot of time to analyze, plan, debate and finally overcome each challenge, not because we wanted to be perfect but just because we didn't want to be wrong in doing any certain part. We as a team are very proud of our accomplishments and how far we have come together, all credits to this amazing hackathon.

What's next for Samaritan

The next step is to take samaritan completely online, drumroll....., and build a secure web app which can be used by the law enforcement agencies to interact with Samaritan. With the help of that webapp and the data we get from Samaritan, we want to make available the following features: -> Tracking criminals in real time. -> Identifying possible suspects based on crime location. -> Finding a criminal in real time. -> Location and timestamp based travel history of persons of interest. and many more.... With this webapp and its ability to make complete use of Samaritan's data, we believe, we can help the government in a way that is technically feasible, ethically correct and thus play our part in making the world a safer place.

Running our code

In the drive folder than has been included below, is the complete face recognition algorithm code. To find out how to run it ,there is another folder inside that drive folder titled ''Jupyter notebook'', which can be simply run sequentially to mimic the process shown in the demo video.

Built With

  • facenet
  • google-colab
  • netgear
  • python
  • tensorflow
  • vidgear
Share this project: