Inspiration
OpenAI created a AI model which gained newsworthiness for being "too dangerous to release." Well, they've been releasing small version of the model periodically, so I wanted to get my hands dirty with it.
The model is commonly known for its creative abilities—the user can give it a prompt and it will attempt to continue the writing, often doing so by creating interesting and coherent narratives. Justifiably, this has caused some people to fear the possible outputs of the model. I wanted to make an artistic statement about this.
What it does
So, I essentially turned this "dangerous" AI into a high school student striving to reach a word count. By carefully playing with the parameters of the model and experimenting with different finetuning techniques, I managed to make the model overfit in a controlled manner to achieve the following function:
$$\text{Given some input text, say the same thing... but a little bit differently}$$
I then fed in an essay that I wrote in high school, and let the model (I call it the verbosifier) get to work and generate a handful of fluff text between almost every original sentence in the essay.
Many of the results were delightfully mundane. Though, despite my abuse, there were still some instances in which the model managed to create some interesting but off-topic narratives.
To highlight this art project I designed a snazzy interactive webpage for people to explore some of the machine generated text. This webpage gives users the opportunity to read the essay with and without the generated text, and lets them change generated passages of the text in real time—allowing users to uncover the expected and unexpected behavior of GPT-2.
Hopefully this page can inspire people to check out GPT-2 and encourage them to try out their own text-generative projects using the model. I also include a breakdown of my parameter exploration at the bottom of the webpage, so hopefully this can reveal some of the inner workings of the model to those that are interested.
How I built it
The interactive webpage was designed by me using HTML, CSS, and JavaScript with notable inspiration from the original OpenAI GPT-2 blog post.
To create the verbosified essay, I utilized OpenAI's pre-trained 124M GPT-2 model, and finetuned it on passages from my essay using Max Woolf's gpt-2-simple package for Python. I then generated samples from the model and hand picked my favorite outputs.
Accomplishments that I'm proud of
I built this webpage so that I could share the results of this art project—the entire webpage is something that I'm very proud of!
Log in or sign up for Devpost to join the conversation.