PencilBuds

Inspiration

Due to the pandemic, online learning has become the norm. Countless students learn by watching video lectures. The lectures are easy to access and offer a quality education for free but are nonstarters for kids with ADHD. The can be long and difficult maintain concentration throughout. Unable to access this resource, the students often be disadvantaged in school. This issue can even negatively impact their mental health.

What it does

PencilBuds is a web app that condenses long video lectures to short notes. It will help students with ADHD do better in (online) school and boost their confidence, providing them the much-needed empowerment

How we built it

We built this app with a React JS frontend with Material UI styling. For our backend REST API, we used FastAPI. This allowed us to use helpful libraries like sumy, which summarizes the content we get from the internet. We use a speech to text algorithm to convert the audio from a Youtube video to text that can be analyzed and summarized by sumy.

Challenges we ran into

We ran into many challenges first in getting the Youtube captions. Initially we used the YouTubeTranscriptAPI library to get the captions from videos provided a video id. This causes issues as some captions did not come with punctuation, causing sumy to not understand how to summarize the text. To solve this problem, we used the Punctuator library to automatically punctuate the text. While this method functioned, it was extremely slow to process.

To speed up the process, we now use the youtube_dl library to download videos, upload them to Google Speech to Text, transcribe them, and return a JSON object to our frontend for display.

What we learned

We were able to learn a lot about the types of natural language processing libraries that Python had. Particularly, we had not heard of the sumy library that summarizes long texts.

Built With

• fastapi
• javascript
• material-ui
• python
• react
• sumy