The current process for creating a visualization for a person of interest requires hiring a professional sketch artist. This process is not only long and expensive, but human bias will always play a role in these illustrations.
What it does
Northern GANs automates this process by generating potential images of a suspect based on a text description. This application can be used to identify a variety of targets including criminals, missing children, and fugitives.
How we built it
We built the backend using Python and Flask, writing various data scraping, collecting, preprocessing and mass storage upload scripts. For the machine learning model, we implemented a novel technique using GANs called Text2Face, using both PyTorch and Tensorflow as deep learning frameworks. As for the frontend, we used barebones HTML and CSS for the most part, directing the user to our landing page.
Challenges we ran into
The dataset used to train the link![Face2Text neural network architecture] was not available to the general public. Hence, at the beginning of the hackathon, we had to message the RiVaL research group at the University of Malta for access to the repository. In addition, the implementation we worked with used both PyTorch and Tensorflow frameworks, which is extremely unusual and difficult to work with. In addition, attempting to train a very complex neural network from scratch was very computation heavy, forcing us to access powerful machines to train our model on, for extended periods of time. Hence, we needed to scale down the resolution of the generative networks' output in order to respect the hackathon's time constraints.
Accomplishments that we're proud of
Being able to implement a novel GAN network architecture and integrate it within our application environment was no easy feat: even obtaining the data required contacting leading academics in Europe to train our GAN. In addition, developing different microservices that would ensure smooth user experiece required high level of abstraction and very modular code.
What we learned
Although we had a lot of fun building our app, training a neural network of this complexity and size requires much more time to obtain acceptable results. We will not be attempting this feat in the context of the hackathon anymore.
What's next for Northern GANs
High dimensional generated images showed tendencies of background noise, which is an indication that our distribution manifold was not completely filled. An obvious remedy for this is to increase the amount of training data. Additionally, we didn't have sufficient time to allow the gan to fully grow. Nvidia's work has shown that a completely grown gan can not only generate accurate images of humans, but can also generate unprecedently high-resolution images (up to 1024x1024)