Noteworthy

Inspiration

(Most of us) take notes on lectures, workshops, and the like. But we realized we almost never use our notes to study and that it's easy to get lost taking notes in a class you're behind in. So we decided to make an intelligent not-taking platform that annotates you notes for you on the go, making it easier to take notes and use them to study.

What it does

We created this wesbite with a customized text editor that allows users to create documents and take notes, save previously taken notes, and edit them. As users type their notes, the site generates annotations on the fly, which users can use to view what they previously said about certain keywords in their notes (e.g. a definition for "Divide and Conquer Algorithms"). Once a user finishes taking a note, the site will also generate and display links for a few helpful KhanAcademy or other educational videos related to the topics discussed in their notes. Users can then go back and view/edit their notes. Users can also create notes by uploading pictures, which are converted to text automatically.

How we built it

Our backend, which computes the annotations, recommends educational videos, and creates notes from images, relies on a Google App Engine standard instance, which we use to compute/store data for the indexed document search api. When new documents are created, they are indexed for future searchability. When a user types, the backend is queried to produce annotations by then performing intelligent, ranked searches on indexed documents (previously taken notes). When a document is saved, the backend performs a keyword search using the Cloud Natural Language API, and then queries the youtube API to find relevant educational videos. The Cloud Vision API is used to convert images to new notes. Our frontend is built on firebase, and is inspired by firepad- an open-source text editor. Notes and annotations are stored in firebase.

Challenges we ran into

Figuring out how to best annotate notes. In terms of what is actual useful information we could return and use to annotate the document without overwhelming the user?
Customizing the text editor to work with annotations and our other requirements.
We has some issues with the functionality with App Engine, specifically limited support for services we wanted, conflicting documentation, and some other small issues.

Accomplishments that we're proud of

We're proud of being able to create a customized text editor and intelligent annotations. We were able to create a custom indexing scheme and search parameters to retrieve annotations, which actually ended up looking helpful for user.

What we learned

Always read the full documentation first. Frontend can be a lot harder than backend.

What's next for Noteworthy

Support for speech to text. Users should be able to also create notes via audio recordings of lectures and the like.
1. More robust file storage and sharing. We'd like to allow users to store files in subdirectories, and securely and efficiently share them with other users.