Inspiration
The idea of the project started from both of us being readers. We've been reading high quality books of authors for years and always wondered how are they able to consistently write at such high quality. I've personally tried writing in the past, and always got a writer's block, I don't know where to start or how to continue beyond an initial idea. For writing exercises online, we've seen simple prompts where they tell you to write about something, but there's no engagement. So we decided to make something that will be able to capture both the interactivity and gamification using a co-pilot.
What it does
The web app we've created uses Azure OpenAI to generate 3 types of prompts for all users daily - normal, challenge, and creative. Our copilot then analyzes the written stories and gives engaging feedback and scores the stories. The score is out of 1000, and our hope is to foster a slightly competitive game-like environment. We have a leaderboard where you can view the best written stories as deemed by our copilot.
How we built it
Our web application was built with Flask, using HTML and CSS for the front end. The application uses Azure OpenAI, Azure Cosmos DB for MongoDB, and more. Our website starts off with a collection of users in our MongoDB database. You need to be logged in to be able to access our main functionalities.
Everyday we generate three prompts per day, one for each of the difficulties. We generate the prompts by using a "gpt-4" model. This is where we implement a "reverse" rag system. We embed each of the prompts with OpenAI's "embeddings" model. For prompt creation we first create a prompt to our specification through system messages, and we give the model all other prompts to make something unique. We then use RAG on this prompt, finding the most similar prompts through vector search. With these most similar we tell gpt-4 to slightly change the content to not be so similar. The reason we did this two step process is because with only one step, gpt-4 still generated very similar prompts as historically, and we found this RAG system obtains the best results.
After the prompts have been generated for all users, users can go to the prompt page. From there users will write a story in a textbox based on the prompt. The user then submits the story, it is then checked with the deep learning library detoxify. If it is above a certain threshold, it is sent to the gpt-4 model to determine whether the content is appropriate or not. If it is deemed appropriate it continues and is analyzed by our copilot.
The copilot is engineered to give a paragraph of feedback based on how well the story is written and how well it follows the prompt, it also picks out two passages it deemed of interest, and picks out two more passages it found has good grammar or bad grammar. The model also gives a score out of 1000 based on our custom rubric. After this is done, the users are lead to an analysis page where all of our copilot's analysis is displayed. The score and story is stored in our Azure Cosmos MongoDB database in an essay collection, so that users can see high rated stories through our leaderboard.
Challenges we ran into
Implementing an Inverse RAG system - Conventionally, RAG system is used to retrieve similar items, however, for us, we wanted to create unique items. So the inverse RAG system idea is to generate an initial prompt, find top 5 similar prompts, then re-generate a new essay different than our top 5 similar prompts.
Daily prompt generation - We had trouble with generating 3 daily prompts for all users, and the solution we came up with is to create a "state collection" which allows us to check for the status of our daily written prompts.
First time making a web app - We've never created a fully functional web app with a proper frontend and backend. Our challenge was mainly to learn as we go, while encountering new ideas and technologies along the way.
Accomplishments that we're proud of
Our accomplishments that we would highlight are -
- Being able to create a fully functional and intuitive web app for users to engage with
- Implementing a unique RAG system that fits the needs of our project
- Developing measures to combat inappropriate content with Detoxify
- Making a feedback and grading system with a custom Co-pilot
- Using new technologies to actually be able to help someone improve real-world skills
- Building a project that can be scaled even further
What we learned
We've learned how you can use these OpenAI models and techniques like RAG in interesting and diverse ways. These technologies we used legitimately helps you get better at writing, there are so many uses cases for these LLMs beyond simple chatbots, and we learned the power of that. We've also learned a lot about web developments and the means it takes to create a fully functional website. We are both students so this project really helped us understand of the scope of what could be done with new technologies.
What's next for Write-It
There are plenty of features that are down the pipeline for Write-It such as your own custom profile page that showcases your essays and scores, checking for plagiarism on stories made by users in the past, and fully deploying the web app.
Log in or sign up for Devpost to join the conversation.