Poster
Workflow

Who

Eleanor Ye (jye20) | Matthew Stephens (mstephe7) (Not like the other guerls)

Introduction

The game design process has historically been a tedious one: from modeling thousands of scene contributions (either in 2D or 3D), to creatively producing unique ways to enhance the objects added to a scene. We are living in a world demanding an increasing amount of realism in the games we play. Additionally, with a correlated increase in computation power, many games of today require a higher level of complexity and intricacy within the objects used to build up the imaginative world of the game. Hence, we present Gamifier: an ensemble program consisting of three deep learning models designed to simplify the game design process. The first of the models is a RNN language model named “Thinker” — trained on both Wikipedia articles and two Harry Potter books — that takes in a user input (say, a single idea for an object to be added to the game) and outputs a list of ideas that are related to the first idea. Once this list has been created, it is fed into DeepMind’s PolyGen model — an “Autoregressive Generative Model of 3D Meshes” — which generates 3D objects as meshes from text descriptions (DeepMind, PolyGen). Additionally, the list of ideas is inputted into a Conditional Generative Adversarial Network (cGAN) — rebuilt and re-architectured following inspiration from the network “Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation” of Bahng et al. (Bahng et al., Coloring with Words) — in order to output a set of color recommendations for the objects outputted by the initial ThinkerRNN model. Through combining these three models, we nicknamed the new umbrella model “Gamifier”, as we believe it will simplify the game design process and unlock a range of potential applications in 3D scene creation, augmented reality, and virtual reality.