We were inspired by Joshua Lafazan speech. He talked about teenage homelessness and how we as a community can use technology to solve some of our society's problems. One statistic stood out to us: 400,000 kids go missing in America every year. In fact, yesterday was Missing Childs Day. We aim to alleviate this problem.
What it does
Our program can be uploaded to any local CCTV camera and perform real-time recognition and analysis of missing persons. If an individual from the FBI Missing person database is seen on the camera, they will be recognized and tracked. The individual's movement will be analyzed, and the program is able to see if the person of interest is walking with anyone (a possible abductor). Finally, the program also generates a map of how far the person of interest could have traveled. This uses machine learning to crowdsource the finding of missing persons by creating a network of CCTV cameras running our algorithms.
How we built it
We split our project into three different problems. The first concerned detecting the person of interest in CCTV footage. In order to do so, we collected training data both from the FBI database and data that we created for each person of interest (POI). The facial embeddings of each training image were then extracted and stored. This was done using OpenCV and OpenCV's deep convolutional neural network library. Next, we trained a support vector machine on the face embeddings and use this to classify every face detected in each video frame. This detected the presence of a person of interest in CCTV footage. The second problem was predicting whether the POI was with a possible abductor. This was done by tracking all the nearest individuals to the POI and calculating their respective distances throughout the clip. Our algorithm used this data in a heuristic to predict whether the POI was with someone. Finally, the third problem was creating a search area map of all places the POI could go in a given time frame. This was done by taking into account the locations population density and public transportation availability (done through finding longitude and longitude of the camera).
Challenges we ran into
The largest challenge we ran into was improving the accuracy of our program. We also ran into many tech issues as one of our member's computers could not connect to wifi, and another's computer was incompatible with some libraries needed. Lastly, collecting training data for our machine learning algorithm was difficult since we had to create the data set ourselves.
Accomplishments that we're proud of
We are proud of the fact that our program is fully functional. It is able to identify a person in a video given a previous image. It can also detect if the missing person is with anyone else. Then with the coordinates of the camera location, it is able to display a map of the possible locations. In order to avoid ethical concerns, none of the data used is available outside of the algorithm.
What we learned
The largest thing we learned was planning. Due to our technical difficulty's, we had to become more efficient with our time.
What's next for MPR (Missing Person Recognition)
What is next for MPR is to get it functional with real databases. Unfortunately, there is no available dataset for missing people, so we had to create our own data. In the future, we would like to get access to real data and run it with our algorithm. We would also like to get the data from people with criminal records to detect if the missing person is with someone dangerous.