Note.io

Inspiration

This project was inspired by previous experience of having to write tedious meeting notes or having to sift through countless videos to find what topics were discussed.

What it does

Note.io is a chrome extension that interfaces with Microsoft Teams (and other platforms) directly to transcribe and summarize meeting notes utilizing the AssemblyAI API. With the click of a button, the chrome extension will automatically begin summarizing the meeting video at a specified Teams folder. From there the AssemblyAI API will return the important topics discussed in the video as well as other information like timing and important words, and a PDF of the meeting notes is automatically posted in the same directory as the video.

How we built it

We build the application using a chrome extension which stores user information client side using the chrome local storage API, and utilized react and JavaScript for the user interface. In order to access Microsoft Teams directly we used the Microsoft Graph API to retrieve OneNote drive files (such as the mp4 files) and then post back the meeting notes back into the corresponding directories. In order to take notes on YouTube videos, we developed a script that would generate a download link for the corresponding YouTube video. To process the video files and retrieve the summarized transcription broken down by topics we used AssemblyAI's Auto Chapters / Summarization feature.

Challenges we ran into

Most challenges we ran into were due to configuration issues with the chrome extension as our group did not have a lot of experience using chrome extensions. For example, installing and using the PDF library with the chrome extension. In addition, generating a downloadable link for YouTube videos was very tricky, since they do not offer that link in their developers API.

Accomplishments that we're proud of

We are proud of building a useful and fully functional application in such a short period of time despite using lots of new unfamiliar technologies like the AssemblyAI API or react. We are also proud of our method of being able to fetch a downloadable link for YouTube videos in order for it to be passed onto the Assembly AI.

What we learned

Our group learned several new tools and API's such as AssemblyAI, the Microsoft Graph API, react and how to build a chrome extension. We also learned how to tackle and solve problems in a short period of time.

What's next for Note.io

The next step for Note.io would be to expand the different services it can interface with such as zoom or other meeting platforms. Another feature to be added is real time audio transcription which is also a feature the AssemblyAI API provides.