Deepfakes, or AI-generated photos and videos, are proliferating across the internet and are very hard to impossible to detect. Deepfakes typically show public figures doing or saying something they didn't actually say or do, at best creating a PR problem, and at worst influencing elections and destabilizing democracies.

What it does

We know blockchain allows us to track the propagation of crypto-coins on a universal distributed ledger. This same utility can be leveraged to track media (photos and videos) as they propagate around the internet. The positive identification of media from trustworthy sources can allow for better policing of deepfakes.

How we built it

Each media item uploaded by trusted sources can be issued a hash code and registered on the blockchain. This hash code is embedded in the media, and is detectable using a browser extension, resulting in any consumer being able to ask the blockchain who first generated the media. The consumer can then make a decision about whether to trust the media or not.

Challenges we ran into

We'd like to embed the hash codes into the media as a digital watermark, but that proved too unwieldy to accomplish during this hack-a-thon, so for now we've settled for storing the hash code as a metadata alongside the media data.

Accomplishments that we're proud of

Handling image data can be really tricky, at first we encountered issues when we updated image metadata and saved the image using one Python package, but then tried to read the edited image with another package. The two packages seem to interpret and store the image data differently, giving us problems for a while.

What we learned

We learned how to create an end-to-end solution for detecting deepfakes on the web. It's a huge problem, and one that requires a strong amount of scale, but we created an MVP capable of searching for embedded metadata critical to identifying whether an image was verified by our blockchain or not, and use blockchain to create an NFT from an image URL. Many of the technologies we used were new to us, but we learned how to be scrappy and build a strong solution in a short amount of time.

What's next for Blockchain to mitigate deepfakes

Work out how to add digital watermarks containing the identifying hash codes into image data that is un-detectable by the human eye, yet robust to attacks. Create a friendly user experience through a browser extension that analyzes images on a website to see if they are verified on pageload.

Built With

Share this project: