web page - home page
mobile app - home page
mobile app - generated page
How I build Demo Website
Revised (UnsplashDowloader) [https://github.com/hating/UnsplashDownloader] to crawl photos on Unsplash website.
Flask server (backend)
A flask server that serves both Flutter App and React website.
React js (Frontend)
React is so friendly and easy to use that help us quickly build a website.
Flutter (mobile app)
A simple Flutter mobile App which allows users to use our generator easily.
GAN ( generative adversarial networks )
methods we have been used but failed
Took Pytorch and Tensorflow 1.0 implementation on Github as reference. We implemented spectral normalization, attention layer, and gradient penalty, and got great results when we generated 64 * 64 pictures. However, the model is too big that we cannot train on our server if we want to generate pictures in size 512*512.
Based on Nvidia 2018 paper, it can generate high-resolution images. We tried to reconstruct PGGAN, but encountered some challenges. We used the concept of transfer learning to build PGGAN, but it led to a bad result. We still needed to overcome these technical problems.
Based on ICCV 2017 paper, instead of end-to-end training solution, we used a two-stage training method to generate high-resolution images from the low-resolution image in an unsupervised way. However, we faced some problem that we can’t get a better high-resolution image by training stage II. Therefore, we still tried to improve the model to get better results.
According to DeepMind 2018 paper, the authors emphasized on generating the various classes and high-resolution images. Hence, we wanted to apply it to our problem. First, we transformed the model and updated the source of the module in tensorflow 2.0. We aimed to reconstruct the result of the paper. Unfortunately, it seemed something wrong with our process and led to divergence on our model. Therefore, we still try to find the error to achieve our target in the process.
Final Model Architecture --> DC-GAN
We adopt the original DC-GAN and modify a bit to solve divergence problem. We babysitted the D loss and G loss to prevent divergence when iterations get higher. Also, we used some training tips on our GAN model and to successfully slow down convergence and stable our training procedure. Finally, it led to fantastic results.
Challenges we ran into
We implemented lots of papers to try to implement STOA GANs which aimed to create high-solution images but most of them failed. Because training GANs is also hard, we normally spent a whole day and get nothing. It was a tough process, For examples, there are some problems we ran into:
- Unbalance between the generator and discriminator that cause overfitting
- Unable to converge
Accomplishments that we're proud of
We not only build the deep learning model that successfully generate images. Also, we build a cross-platform App that fit users from different devices can use our product. Also, we did lots of jobs focusing on GANs training, although, it does not always work, we learned a lot .
What we learned
We have learned how to create an end-to-end deep learning product from scratch. Build a product is never an easy thing, but we did it. Also, we are more familiar with TensorFlow 2.0 API, which just makes a big change but becomes more easy to pick up!
What's next for Amazing Signts
We are going to try different model structures to generate more realistic and high-resolution pictures. At the same time, we will continue to collect more data and try to generate synthetic images as more types as we can. If we can create more realistic and high-solution pictures, Amazing Signts can easily meet everyone's need.