Shakespeare LSTM

LSTM trained on 32 plays to create its own Shakespearean works

Comment

Inspiration

I've made an LSTM before but ended up with a net that didn't perform as well as I hoped. I took this hackathon as a learning opportunity to get experience with Tensorflow and a chance to try LSTMs again. I did change what that net trained on: Shakespeare plays instead of haikus.

What it does

The net trained on 32 plays by William Shakespeare to be able to output — as a .txt (and optionally a .pdf) file — an arbitrary amount of content meant to mimic his works.

How I built it

As should be with entries in this hackathon, this project relies on Tensorflow 2.0. This tutorial was very helpful. I made multiple versions of the LSTM to try out various approaches to the text generation. The first and second versions analyze the patterns of punctuation marks and entire words in the training data and use those as their vocabularies for text generation. The first version groups all words that appear less than 50 times into one item in the vocabulary: "" (for unknown). The third and fourth versions analyze the patterns of singular characters in the training data and use those as their vocabularies for text generation. The fourth version has many more neurons than the other versions. Under my training, version 3 produced the best results. It is the author of the example text on the GitHub page, and the images on this post showcase excerpts from that same file.

Challenges I ran into

At some point, versions one and two became much worse for a reason I have not yet identified. They both started to refuse to include spaces, newlines, and tabs in their output at any point in their training. I did not prioritize finding the cause of this issue, as version three is the best version regardless.

Accomplishments that I'm proud of

I am proud of the improvement I made in my second attempt at an LSTM project. I am especially happy with how quickly the net picked up on the formatting used in the training data: all caps for character names not in dialogue, appropriate whitespace, capitals at the start of each line, punctuation that makes sense, etc.

What I learned

There is no doubt that this project helped familiarize me with Tensorflow and LSTMs (and let me brush up on my Python as well!).

What's next for Shakespeare LSTM

I would like to find the cause of the issue where the first two versions are not including whitespace in their output. After that, I want to try to increase the complexity of the first two versions to see if that will work better with their larger vocabularies.

Built With

Share this project:

Updates