Many online learning platforms have quizzes integrated into them. However, these quizzes are made manually by the instructor or organization. Furthermore, a very common study method employed by students is to create quizzes/flashcards for themselves. This lead our team to wonder if there was a way to create these quizzes automatically, taking the time away from making these quizzes and freeing it up towards other activities such as more studying. Thus, the idea for Quiz It! was born. After all, it is said one learns best by doing.
What it does
Our project lets users upload a photo of their notes to our server. From there, the note's contents are parsed and relevant questions are made with appropriate answers. These question sets are then uploaded to Quizlet as study sets for the student to study whenever they wish. On the other end, we developed an Amazon Alexa skill to interactively quiz the user based on any of the study sets stored under their Quizlet account. This allows the user to simultaneously study while completing other tasks such as laundry or cooking, leading to more overall study time.
How we built it
There were three major parts to this project: Amazon Alexa, question creation, and Quizlet integration.
Alexa was created using AWS Lambda functions and the Quizlet api interface. Communication with Alexa is sent to an AWS lambda function in JSON form, where each 'intent' is further parsed and analysed. Because lambda is stateless, session information is passed from lambda to Alexa, which returns that information on the next request. In essence, state is stored by the Alexa. Within the skill, the user is led through a series of intent endpoints that query the Quizlet API to retrieve user information which is used to test the user.
Creating questions involved taking the picture provided by the student and analyzing it for text using Google's vision API. The extracted paragraph was then tokenized using Google's Cloud Natural Language API and converted into word objects, including descriptors of its part of speech. Entities were then also extracted using Google NLP's entity extraction capabilities. This allowed us to identify the more important parts of the sentence (salience), more information about its word type (person, place, etc), and even have information about its Wikipedia page. However, pronouns in many sentences needed to be dereferenced. We assumed that the sentences would follow the subject-verb-object format standard in English, and thus identified the subject as the first occurring noun or pronoun. Each pronoun was then switched with the subject of the proceeding sentence. We then Identified sentences containing certain keywords to be converted into a fill-in-the-blank question format. Each "blank" was decided based on the value of the salience of that entity. By this point, we would have a list of tuples containing a question and its answer, ready to be sent to Quizlet.
Quizlet integration involved using the platform's API to get the study set data for Alexa, and uploading the questions and answers produced to create new study sets under the user's account.
Challenges we ran into
In order for Alexa to be robust, it must be provided with many good examples of a possible answer to train its neural networks. Developing the back end of our web app was also a challenge as our team did not have extensive experience in that area. As well, developing a good algorithm for creating proper questions took some time.
Identifying how to translate paragraphs into fill-in-the-blank questions also required a certain level of linguistic knowledge which we did not have. As such, we have had to frantically learn some language arts in the middle of a technical hackathon.
Accomplishments that I'm proud of
After much trial-and-error, we are able to have the Alexa interact naturally with the user, and successfully go through the Quizlet study sets. We also encountered some trouble in developing the back end of our web app, as the team did not have extensive experience in this area. However, we were able to persevere can develop a functioning backend.
Questions are also being successfully generated using only the picture of a segment of text, such as notes or textbook pages.
What we learned
Programming a voice assistant, natural language processing, and backend development.
What's next for Quiz It!
In the future, the project could expand to a mobile app and could reference many more external sources to develop questions with improved sophistication.
In terms of the natural language processing component, a much smarter way to dereference pronouns would be using the Standford CoreNLP's coreference resolution system, which can use deep neural nets to link pronouns and the nouns they reference.
The questions themselves may also be changed to a multiple choice format, where similar words to the actual answer are identified using a word2vec corpus.
Alternatively, the questions could also be changed to be a more natural English question, containing words such as who, what, when, why. This would require a more robust system which is able to naturally rearrange parts of a sentence to form a question.