What it does
We created a Discord bot that learns to write like you and your friends using the latest research in AI and machine learning. Simply, add the bot to your discord server, choose the user you want the bot to act like, and then an AI will be used to generate realistic messages.
How we built it
We used Discord.JS and the Discord API to design the "Imposter GPT" bot, which is the primary interface for our project. We developed the backend on Google Cloud using the Compute Engine. We trained GPT-J, one of the largest NLP language models, using Cloud TPUs from Google's TensorFlow Research Cloud (TRC). With these resources, we were able to train GPT-J incredibly quickly.
Challenges we ran into
We ran into several challenges working with GPT-J and quota limits on the Google Cloud Platform. The model is so large that many of the standard approaches to hosting a machine learning model in production are not suitable. We fixed this by using a similar and smaller model, GPT-Neo, for quick responses. However, we still manually trained GPT-J using v3 TPUs on the message content for a specific server so we could test it.
Accomplishments that we're proud of
We're proud of writing a discord bot that uses AI to impersonate users in a server. Furthermore, we're proud that we learned about Discord bot creation and how to use various Google Cloud products.
What we learned
We learned all about how to create a Discord bot and link it to an external API that we wrote. We learned about various services on Google Cloud such as the Compute Engine and TPUs for training machine learning models. Importantly, we learned that getting a minimal viable product as fast as possible is of utmost importance during a hackathon. For the AI portion of the project, we started with a large model that was tough to work with, but instead we should have started with a smaller model that would have been easier to work with and eventually worked up to the large models.
What's next for GPT Impostor for Discord
If we were to continue this project in the future, we would automate the process of fine-tuning GPT-J. We started writing code to automatically perform this training, but we ran out of time to get it fully implemented. We focused a lot on enabling our training and inference solutions to scale, but this wasted a lot of time in the beginning. If we had more time, we would have been able to implement these advanced automatic scaling techniques. There are also some additional modes we would like to add to the Discord bot.