As piracy and "deep fakes" become increasingly prevalent, we wanted to find a way to robustly and resiliently encode digital signatures into media. Typically, digital ownership is verified using some external digital rights management tool, which becomes useless after the media has been decoupled from that application. By encoding signatures directly into the media itself, our system tries to bypass this issue, and offers a method for reliable authorship verification.

From a technical point of view, we also wanted to explore ways to harness small perturbations (typically associated with adversarial input) for productive use. An adversarial example is an input created by an attacker which is specifically designed to trick a machine learning model to generate false output. Typically, they are viewed as a weakness of deep learning, but we propose that they can also be used as a method to encode images with messages that are extremely resilient to transformations and abuse.

What it does

Deep neural networks such as VGG16 and ResNet101 have been used to achieve state-of-the-art results in image classification. We combine these pretrained deep neural networks — whose parameters were expertly honed to detect salient shapes and features — along with stochastic connections and layers to develop a “decoder” model. This decoder acts as a robust, cryptographically-secure, transformation-invariant hash function for images, mapping input images to 32-bit codes.

By performing projective gradient descent on the decoder model with respect to a given image, we can use it to “sign” images robustly. We start with the original image, then repeatedly tweak the pixel values such that the image (and all transformations, including scaling, rotation, adding noise, blurring, random cropping, and more …) decodes to a specified 32-bit code. The resultant image will be almost imperceptible from the original image, yet contain an easily-decodable signature that cannot be removed even by the most dedicated of adversaries.

We apply this to the problem of giving creators a unique way to provide proof-of-authenticity for their work. In the website we developed, we enable users to embed Facebook profile IDs in images, and decode images to find the Facebook IDs of the people who authored them.


The applications of this kind of technology are endless. Perhaps most saliently, it offers an almost-foolproof way to detect piracy. Even if pirated content is altered in exotic ways, through cropping, adding text, translations, color distortions, or more, the deep neural networks we developed will be able to detect it. It offers proof of authenticity for image content, providing a safety barrier against use of deepfakes/other video faking technology to stage hoaxes. It allows consumers to easily track down the creators of the content that they love. We could even use this tech to eliminate the QR code.

Built With

Share this project: