-
The notetaker component in action
-
Subtitles component in action
Inspiration
To me, education is about knowledge sharing and a good portion of this happens in the classroom. I wanted to make something that would equip students with high-quality notes and improve our ability to learn. Furthermore, I wanted this web app to be accessible to as many people as possible. Since there is a delay in the text processing (due to backend tasks), I added another component for live pseudo-AR subtitles so students with limited hearing can get more out of the application.
What it does
Notetaker transcribes audio but also bolds keywords and adds images to the notes. It also has a component for AR subtitles using your phone.
How I built it
The backend is done with Python's Flask. For the front-end, I used HTML, CSS, JavaScript, jQuery, and Bootstrap.The NLP is done with NLTK, specifically using the RAKE algorithm for keyword extraction. The images are retrieved from ShutterStock's API.
Challenges I ran into
- Building a good front-end is tough!
- Keyword extraction works best on long bodies of texts so using the algorithm on short sentences was tough
- Setting up the phone camera stream was interesting. I actually used recursive jQuery! D:
- I tried using the GrammarBot API to add punctuation and capitalization to the notes, but the suggestions were odd sometimes.
- Tried to play around with libraries such as fastText to be able to do dank machine learning and natural language processing but couldn't figure anything out :(
Accomplishments that I'm proud of
It's fun saying random words and seeing what stock image ShutterStock's API will return.
What I learned
Learned a lot about Flask, NLP, and front-end dev
What's next for Notetaker
It would be cool to add more NLP such as summarizing sentences and keyword extraction with multiple words (such as 'water bottle'). Also, it would be nice to add punctuation and capitalization.
Log in or sign up for Devpost to join the conversation.