Alexa Tutor

Inspiration

You know how you love a course but still, there are moments where you totally phase out and wake up to realize you have missed some crucial stuff, or there is a reference to the last lecture which you understood well but you just a need a quick revision of what the Professor said in that lecture. This app sorts all these problems with even more functionalities. Alexa tutor takes notes in every lecture and displays in real time on the student's laptop. You can search for keywords and phrases in these notes. You can also email these notes to yourself. Additionally, this tutor uses intelligent machine comprehension to answer any questions and if you are not satisfied by Alexa tutor's answer, you can send the same questions to the Professor with one click.

How we built it

We built it in Python Language taking advantage of both Google Cloud computing and Amazon Web Services. This application has two modules -- one is meant to run on instructor's computer which records the instructor's voice using Google speech2text routines and sends the speech text to Amazon Web service. Meanwhile, each student runs the second module of this application. This module keeps in touch with Amazon web service when it finds new information it updates the information on GUI accordingly. While making user interface we faced trouble in using multithreading effectively with QtGUI framework. In future work, we plan to make it multithreaded so that user does not have to manually update the new information.

Challenges we faced

We tried to make an Alexa skill but it seems it is impossible to do it without being able to code the Echo hardware directly (since the hardware is Amazon's proprietary). While we are not sure if a third party developer can make this entire app as an Alexa skill but we can bet an engineer working at Amazon can make this complete app.

Accomplishments we are proud of

We are quite proud of the fact that we got every part (listening, communicating and reading in real time) working in less than 24 hours!

What we learned

We learned about the various voice APIs available online and interesting speech-to-text conversion methods (the one that worked best was using Neural networks). We learned how to use Amazon Web Services to communicate between two computers and how to develop a GUI which updates its contents in real time. A lot of this helped us understand using multi-threads while programming.

What is next for Alexa Tutor

In future work, we can use efficient machine comprehension to understand the notes and answer more complex questions. We can integrate this with echo look which takes snapshots of the blackboard and displays it along with the notes to refer to diagrams. We can also add a Wikipedia or textbook search functionality in the app.