verifEye

A fast and lightweight utility to detect online identity theft and fake news using machine learning

Comment

Inspiration

We live in an era of misinformation - on social media outlets, fake news now spreads faster than the truth. Worsening this problem, recent developments in AI have enabled the creation of incredibly realistic fake videos generated through deep learning - so called "deepfakes".

What it does

verifEye analyzes abnormalities in eye movements and feature rendering using a single feedforward neural network. This solution avoids using computationally expensive CNN structures, and enables rapid analysis by cutting out most of the computational expenses using heuristic methods. Our model effectively flags videos that appear suspicious using a lightweight neural net model. The output of the model is funneled to the user through an extension (currently implemented for chrome).

How we built it

Whereas normal methods for computer vision analysis function based on CNN architectures, our model uses smart heuristics to undercut most of the computational strain. This allows Tensorflowjs, which runs entirely in the browser, to compute and predict using small feedforward models that can actually execute using very little power.

The model is simple:

const model = tf.sequential();
model.add(tf.layers.dense({units: 2500, activation: 'sigmoid', inputShape: [64*64]}));
model.add(tf.layers.dense({units: 2500, activation: 'sigmoid'}));
model.add(tf.layers.dense({units: 2500, activation: 'sigmoid'}));
model.add(tf.layers.dense({units: 64*64, activation: 'sigmoid'}));
model.add(tf.layers.dense({units: 2, activation: 'softmax'}));
model.compile({optimizer: tf.train.adam(0.00001), loss: 'softmaxCrossEntropy'});

This is thanks to the work that is undercut thanks to heuristics:

//Blur gradients
FOR(pixels in frame):
    GET(variance of laplacian in neighboring area)
    GRADIENT(variance)
RETURN(resulting matrix)

//Eye analysis
FOR(frames in video, sampling every n variable frames):
    GET(EAR(frame)) => data
    IF(data > THRESH):
        //register blink
        BLINK++

//ELA
FOR(i in range(var n)):
    COMPRESS(frame, var strength) => frame_
    frame -= frame_

Challenges we ran into

Implementing Tensorflow with JavaScript and running async code has been a real challenge. The styling on our website was also the most advanced that we had ever done.

Accomplishments that we're proud of

Apart from the Chrome extension itself, we also created a presentation website. It took a lot of careful designing to make it. We are especially proud of the animation effects on the site.

What we learned

We learned a lot about styling and javascript. Pertaining to the optimization of tensorflow in javascript, we learned about the creation of effective heuristics, and also learned to appreciate how fast google cloud could work when used at full power!

What's next for verifEye

Our system empowers streaming sites and users alike - if utilized, a streaming site could rapidly assess incoming uploads for evidence of deepfaking with negligible delays. Our system would also enable quick, computationally light personal use.

Media distrust is now higher than ever, thanks in large part to the continued emergence of hundreds of sites which make exclusively fake news. Now, more than ever, we need a tool to help us distinguish the truth from lies. Our solution will help websites and users alike avoid falsified media which otherwise could reach hundreds of thousands of unwitting people before being caught, if at all. We hope that our work is a step towards a safer, better-informed future.

Built With

Share this project:

Updates