As a group, we were looking for a way to help people in sort of an unexpected way. After thinking about it for quite some time, we realized that a large majority of students in college have never developed the proper note taking habits that are required to get every single important piece that a professor may talk about. Some facilities on campus offer note taking services; however, this is inconvenient to a certain point because the student has to rely on someone else for them to succeed in class. This product completely removes any issue that students may have with taking meaningful notes in class, and allows them to keep track of every important detail that their professor touches on.
What it does
TL;DR: our app records what a professor says and organizes a study guide based on the most important parts of the lecture.
Our mobile app utilizes voice-to-text to track what a professor is saying in real-time. After a class has finished and all of the speaking has been converted to a manipulable data type, we send the words that were just spoken to a flask back-end that was deployed on Heroku. The server processes the words using artificial intelligence to predict where the punctuation should be. The API that is being used, to create our tags (highest occurring words) and the summary of the text, requires punctuation which means that the only way to place punctuation was by following a heuristic model to estimate where periods, exclamation points and question marks need to go. After the words were formatted in a proper grammatical way, the most important points of the text were organized into a document to be viewed in the mobile application or on the web app that was created specifically to support the crowd sourcing of different lecture notes.
How we built it
We built this project by using an android based mobile app, which communicates with a flask back-end for all of the data processing. We built the web client with vue.js and took advantage of firebase for all of the data persistence.
Challenges we ran into
Voice to text does not take punctuation marks into account without explicit declaration. Because of this, we needed to use a prediction model to determine where sentences end in order to process the meaningful sentences properly.
Accomplishments that we're proud of
We are extremely proud of the way that we created an entire detailed study guide for people to use based off of something as simple as a professor speaking.