Movie Title and Overview Generator

Inspiration

We really like watching movies, and we were also really interested in language models and wanted to see if we could build our own from scratch. Therefore, we decided to fine-tune a language model to generate new movie titles and plot summaries based on user inputted genre, keywords, and uniqueness.

What it does

Given a list of genres and keywords along with a chosen scale from genericness to uniqueness(0-100) and a model choice(T-5 small trained on 230,000 points or T-5 base trained on 20,000 points), our language model will output a movie title and overview. Here is an example input and output from our model:
Input:
Genre: Sci-Fi, Action, Adventure
Keywords: space, aliens, technology, labs, future

Output:
Space Shuttle
The future of the USSR is now run by a two-year-old scientist, as he struggles to stop the aliens.

How we built it

We used a pre-trained model from huggingmodel's transformer library. Specifically, we used Google's T5 large language model. We fine-tuned it using by giving it example inputs and outputs from a dataset of movies from The Movie Database. We then manually tested different hyperparameters to further improve the output to sound natural and more importantly output interesting and unique movie summaries instead of generic ones.

Challenges we ran into

Our biggest challenges are long run times and limited experience with building language models. We overcame them with a lot of help from online resources to self-teach ourselves the basics during the duration of the hackathon. For long run times, we optimized our device to utilize NVIDIA GPU.

Accomplishments that we're proud of

A working language model that generates natural sounding movie summaries and titles. You can imagine some of these could be real movies!

What we learned

Some basic theory of how language models work, how to fine tune an LLM, how much compute it takes to train LLMs.

What's next for Movie Title and Overview Generator

Use larger models fine tuned on more data. Our best model uses T5-small and takes hours to fine tune. We also built a T5-base with 20,000 points. With more time and compute, it should be possible to use a model with more parameters such as T5-Large and train in more data points.