Inspiration
We attempted to implement a DCGAN (https://arxiv.org/abs/1511.06434) from scratch to test out the feasibility to use PaddlePaddle as an efficient prototyping tool for experimenting with deep learning architectures in research.
What it does
Provided that the training of DCGAN involves two deep neural networks that compete with each other, we think implementing both the model and the training process will give us a better insight into the tools that the PaddlePaddle platform has to offer.
How we built it
We relied largely on the tutorials and API documentation that the Baidu team had prepared for the event. The model definition is convenient and easy to grasp. With a few simple helper functions, we managed to define both the generator and the discriminator with less than ten lines of code.
Challenges we ran into
The trainer provided by the current version of the platform is very well self-contained. While it provides a fast and easy solution for defining the training process for single neural net architectures, we found it difficult to break down the trainer and take care of both generator and discriminator. After exploring different implementation strategies including priority (prioritized by loss) based round robin training and introducing a separate discriminator to help separate inference and weight update, we cannot find a good solution to implement a trainer for GAN.
What's next for DCGAN in PaddlePaddle
We did not have the time and resource to experiment with the scalability of PaddlePaddle, which makes it rather suitable for production environments. We also became aware that GAN will have a better support in the near-release new code of PaddlePaddle. We hope to find a cleaner trainer implementation when the new code is released.
Log in or sign up for Devpost to join the conversation.