DeepGuard

Inspiration

DeepGuard was inspired by the ever increasing use of deepfake technology on modern social media platforms. As kids, many of us spend hours a day on our phone consuming content from social media companies like Facebook, Youtube and TikTok. Many creators whom are watched by millions of kids around the world fall victim to misinformation that often come in the form of a devious technology known as Deepfakes. Deepfakes can impersonate people to almost a scarily accurate degree and make them say things that they normally might not have said. We as kids, almost fell victims to these traps when we were young and so in an effort to prevent the same trend happening to future generations, we hoped to build a project that stops this ordeal once and for all.

What it does

DeepGuard is a novel solution to detecting deep fake take a multimodal approach to identifying and warning users about deepfakes. DeepGuard employs a novel detection algorithm that parses three pieces of crucial information from a Deep Fake video: Video display, Audio and speech analysis. By parsing this crucial information from a Deep Fake video, DeepGuard then runs a custom novel detection algorithm that breaks down information and tries to identify key features that might indicate that it is a Deep Fake.

How we built it

We built DeepGuard using a react front end combined with a FastAPI backend. Our novel detection algorithm was ran through various API's and ML algorithms. For voice detection or voice spoofing analysis we used a machine learning algorithm from the internet known as a Resemblyzer which is trained on a voice spoofing dataset and our own custom voice samples to identify fradulent or AI generated voices. For Video analysis, we used Arya.AI, a Deep Fake detection API catered specifically towards video analysis. Lastly for speech analysis we combined to API's plus a little bit of prompt engineering. We used google's Chirp Speech to text API to get the text of what was spoken in a DeepFake Video. We then fed that text to ChatGPT 4.0 API and asked it to cross reference that text by what was typically said by the subject in the DeepFake.

Challenges we ran into

Our biggest challenges we ran into was prompt engineering for ChatGPT to give us the custom data we wanted it to. We spent almost 12 hours creating various prompts for ChatGPT and Intel's Developer Cloud LLM to be able to take the text of a deepfake and compare to what was typically said by the subject in a DeepFake. The output was often not specific or confused about the desired outcome and the unpredictability of the LLMs made it hard to build a prompt that fed us consistent data.

Accomplishments that we're proud of

We are proud of the UI and implementing a completely new backend in python which we had never done before. We are also extremely proud of finally being able to integrating machine learning and LLMs into our project which we had struggled to do many times before but were finally able to accomplish it well during this project.

What we learned

We learned a LOT about prompt engineering and LLM's. We also learned a lot about using various API's and custom machine learning algorithms along with FastAPI. On the frontend we learned a lot about various react elements and creating creative UI's.

What's next for DeepGuard

DeepGuard is only starting. In the future we hope to integrate it with YouTube to be able to analyze a wide variety of videos. Additionally we would want to integrate image analysis on top of the video analysis we already have. We also want to build custom machine learning algorithms in the future besides relying on prebuilt API's