Deepfake Shield

Overview
Grid Search for Hyperparameters using MLRun
Data Preprocessing using MLRun
Training Model using MLRun
Prediction Pipeline using multiple models
Deployed web-app: https://deepfake-shield.herokuapp.com/

1. What is a Deepfake?

The term “Deepfake” is referred to a deep learning based technique that swaps the face of a person with another face in an image.

2. 📗 The Problem

It is easier than ever to create deepfakes of anyone using the tools available online. Deepfakes can be used by people to generate fake news, hoaxes, blackmailing, financial fraud, and many more malicious activities.

3. 💡 Our Solution

Deepfake Shield is a tool that uses deep-learning to detect deepfakes in an image. The diagram below summarises our project. Feel free to try out the web-app - https://deepfake-shield.herokuapp.com/

4. 🧙‍♂️ Making the most out of MLRun

Our favourite way to use mlrun has been the # mlrun: start-code and # mlrun: end-code. The ease of use in terms of tracking experiments helped us progress rapidly from experimentation to training and deployment, all without the hassle of trying to keep logs manually.

4.1 Data Preprocessing

When preprocessing the data, mlrun.artifacts.PlotArtifacts helped us visualise a bias in the dataset. We found that the number of real images is much lower than that of the number of fake images. This was fixed by inflating the number of real faces using the FFHQ dataset

4.2 🔬 Automated Hyperparameter Search

Finding suitable training hyperparameters manually can be quite tedious. We automated this process using mlrun.new_task().with_hyper_params(grid_params, selector="min.loss"), thus making the process of finding hyperparameters a lot less painful.

4.3 💡 Training + Evaluation

The pipeline that we built comprises of 2 different models:

We use a pretrained BlazeFace model (which can be retrained if needed) for extracting faces from images.
We trained a customized implementation of EfficientNet for classsifying the extracted faces accordingly.

The model was trained using the ideal hyperparameters found using grid-search with mlrun. The training and evaluation logs were also tracked using mlrun

📗 Resources

Data exploration: NBViewer, Github
Preprocessing + Hyperparameter search: NBViewer, Github
Training + Evaluation: NBViewer Github
Dataset: We've used a modified version of the deepfake-detection-challenge dataset.