Inspiration

We were keen to do something fun during the Hackathon. We have experience with Machine Learning using surrogate modelling and time series, but rarely work with image data. Therefore we wanted to do a project that would take us outside of our comfort zone to work with GANs and image data.

What it does

The project is an online game board for Dungeons and Dragons that allows participants to play together remotely but share the experience. The Dungeon Master has control over the environment, where they can describe the next scene that the players will encounter. This will trigger the generation of new images by the Generative Adversarial Network working in the background. The game will read the Dungeon Master's input description and identify the creatures they are describing. It will then generate new game scenes that are representative of these creatures to share with the players.

How we built it

The front end of the project was constructed with Unity. The back end works using Python, utilising trained models such as bigGAN and a pretrained word vector dataset.

Challenges we ran into

We started off attempting to train our own GAN on custom datasets that we searched for online. The main challenge we faced was finding large enough datasets to work with for a GAN, especially those with labels. We also had issues finding sufficient compute resources to train the model efficiently.

Accomplishments that we're proud of

We are proud that we were able to produce a working program that would take a sentence and create a creature out of two described characters.

What we learned

Balancing the generative and discriminative loss in a GAN is a black art that will take more than 2 days to master.

What's next for DunGANs and Dragons

A more intelligent text reading model and a more interactive UI, as well as using a GAN to generate images for the environment.

Built With

Share this project:

Updates

Submission history