With artificial intelligence becoming a handy tool for almost everyone, even those who do not understand machine learning, numerous aspects of our lives have been deeply shaped by such technology. Deep learning with neural networks has greatly empowered fake image generation and even art. In HackPrinceton 2019, my teammate and I endeavored to create a generative network that translates simple sketches to landscapes through adversarial training.
What it does
Any simplistic drawings can become an artistic landscape. The handwritten traces are processed through a convolutional neural network, and it outputs a stylistic landscape based on the drawing.
How we built it
We used the GAN model proposed as Pix2Pix, which performs a pixel-level transformation. The model construction and some training scheme are adopted from Image-to-Image Translation with Conditional Adversarial Networks from ICCV 2017 by Jun-Yan et al. The dataset is originally from Kagel, and we applied heavy modifications to make it suitable for pix2pix GAN.
Challenges we ran into
It was very time consuming to go through all of the original code and try to adapt parts into our code. It was also quite challenging to find a proper way to label all of the data since there was only a raw dataset that needed optimization. The most challenging difficulty is a hardware issue that we cannot get around. Without a video card, using a CPU for machine learning makes it impossible to run through all intended epochs.
Accomplishments that we're proud of
We came up with the idea at the end of the day and in the end, we did manage to successfully format the dataset and set up the training for the neural network.
What we learned
It is the first time that we have tried to create a whole dataset for deep learning with automated tools. It is also the first time for us to take theoretical proposals on paper to actual projects.
What's next for Deep Landscape
Our next step is to refine the training dataset and testing dataset and run the project on a platform with better computational power.