TextAssess

• 

  Architectural Diagram

Inspiration

The inspiration behind this project is to address the challenges faced by students and teachers when it comes to writing and grading written responses. By leveraging the power of NLP technology, the project aims to provide a faster and more efficient way to grade responses, while also helping students improve their writing skills.

What it does

This NLP architecture takes in user input of a prompt and a response. Then, using common Language models, the user's response to their prompt is rated on similarity to the context of the question (judging how 'well' the user answers the question). Then, using an alternative (generative) model, a ML-generated response is concocted. At this point, we have two possible outputs to the same prompt. Therefore, using language models, responses can be compared, and better sentences can be chosen at each stage (between the user's sentences and the computer generated sentence).

Models used are Google's BERT model and OpenAI's GPT-2, hosted on HuggingFace and fine-tuned with AWS SageMaker.

How we built it

To demonstrate how this model works, refer to the architectural diagram below. It incorporates our future plans in addition to what is currently implemented. At the moment, our output represents how well a user answered a prompt with the metric being a similarity score between their response and a generated response. We are using cosine similarity to accomplish this by passing embeddings of the responses into the scikit-learn cosine similarity function. Then, we are identifying which parts of the user's response are not good enough.

Challenges we ran into

We used tools and platforms such as AWS Sagemaker and Hugging Face for the first time. There was a lot of debugging and reading documentation involved but we were able to get a prototype working using these new tools as well as our previous expertise.

Accomplishments that we're proud of

Submitting a working prototype with the ability to be improved and have a real impact.

What we learned

Working with an end-to-end machine learning pipeline and utilizing cutting-edge technology to solve a prevalent issue.

What's next for TextAssess

Fully implement the "improvements to response" feature and create a user-facing product such as a website or app so students and teachers can easily access and benefit from all the features.

Built With

• amazon-web-services
• google-bert
• gpt-2
• hugging-face
• jupyter-notebook
• python
• sagemaker
• scikit-learn