Deepfakes are hyper-realistic synthetic media developed using Artificial Intelligence, for various purposes- but mainly malicious purposes, like undermining journalism and people's trust in democratic government institutions, harming the reputation of people (eg: revenge pornographic deepfakes) and spreading fake information to widen gaps in societies to cause riots etc. Deepfake are rising at a rate of 6280 times (Year-On-Year) and are reaching people 6 times faster than real news. This is a growing threat and thus it is important to counter them by developing tools that can help us detect the, and prevent them from spreading. This inspired us to work towards solving the problem, and so we developed Detectd.
What it does
Detectd uses the same technology to detect deepfakes that is used to create them- Artificial Intelligence. Detectd allows the users to upload video to our platform and check them if it is a manipulated video or not. This will help people and institutions verify the authenticity of video that they use or share to the public, and help counter the bad actors who rely on spreading fake news/information with malicious intents. Detectd helps counter the misinformation warfare.
How we built it
We used Python, its multiple libraries and Tensorflow framework to develop a Convolutional Neural Network combined with Long Short Term Memory Network to analyse 40 frames from a video in a sequential manner. While we used Flask to deploy the model in a WebApp and serve it to the public through Azure App Services. Our CNN was pretrained on ImageNet dataset and we further trained the entire model on FaceForensics++ and DFDC dataset. This has made our model more robust- it can now detect variety of manipulated media- Deepfakes, FaceSwaps, Face2Face and even Neural Textures.
Challenges we ran into
We ran into a some challenges not while developing but deploying the project. We are using the videos uploaded by the user as input. So sending the videos from front-end to back-end in a manner which is fast, secure and efficient we used the Base64 encoding. Also, one of the libraries we are using is OpenCV and the deepfake detection model is developed using Tensorflow. So while deploying the project to Azure App Services, we faced trouble due to a specific library- LibGL.so.1, which was required for OpenCV to function. The problem did not arise on our local environment (Windows) but in the app deployment container (Linux). However, we solved this using SSH- we installed the libraries manually, remotely, and also edited our code to ensure that it works everytime. Another issue we faced while deploying the model was that the model which was stored in the TF SavedModel format could not be loaded and hence could not be used. We solved this error to by making changes to our code that force loads the model at startup, before the first user request.
Accomplishments that we're proud of
Multi-tasking. We implemented a complicated Convolutional Neural Network - Long Short Term Memory Network within 4 hours from a research paper. Then we trained it on high quality deepfake data and ultimately deployed the model to Azure App Services. But we are most proud of deploying our model for the public use. Since this was our first time deploying a model.
What we learned
We learned how to use Flask and Microsoft Azure App Services for deploying a deep learning model for public use- accessible through a url.
What's next for Detectd
We plan to add the opportunity to detect manipulated pictures and audio to make it a one-stop deepfake detection solution. Ultimately we plan to make it available to police- cyber crime cells to combat the biggest cyber threat of 21st century.