Facial Composites are traditionally drawn by sketch artists, it’s a graphical representation of one or more eyewitnesses' memory of a face. This is a slow and an expensive process because hand-drawn rely heavily on average witness’s imaginative abilities, making accuracy harder to achieve. This makes it harder for perpetrators of crimes to be caught. We wanted to use advances in machine learning to help investigators catch these perpetrators and issue justice to victims.
What it does
Deep dreamer is based on a generative adversarial network that "imagines" ultra-realistic faces of suspects based on feedback and responds to witness's feedback to re-generate variations until it co verges on a best-match image.
How we built it
Based on StyleGAN, a novel generative adversarial network. It works by taking an arbitrary latent vector z, learns a fully connected mapping from z to an intermediary feature vector w, and then learns a mapping from w to images of faces via convolutional layers in a synthesis network.
Our main challenge was to map machine features to features humans can understand. We trained a Fully Connected Neural Network to learn a mapping from real features like race, ethnicity, age, hair color, to machine features (in the latent vector z). We took average of all faces’ latent vectors to generate the gradient to be added to allow witnesses to adjust features like masculinity, ethnicity and age and create indistinguishable faces.
Challenges we ran into
Managing dependencies on systems and developing a technique to understand the relationship between human attributes to machine attributes.
Accomplishments that I'm proud of
Allows investigators to create high resolutions and ultra-real images of suspects based on witness descriptions. State of the art generative models indistinguishable from real faces. Developed a detailed transformation from human features to machine features
What's next for Deep Dreamer
Integration with natural language processing models to record witness testimonials. Reach out to police officials to attempt a trial of this system, extending its implications beyond the hackathon.