Inspiration
With the rise of recorded lectures due to the pandemic, students have an abundance of recordings at their fingertips. The issue is with so much data, it becomes hard to quickly review or figure out what lectures contain what info. That's where Recap Bot comes in!
What it does
To use Recap Bot, just drop in a video to start. From there, it transcribes it, chunks it into different topics, and then summarizes these chapters. From this, the user can easily scroll through either the transcript, or the summary and click on the timestamps to easily jump in the video player. Recap Bot is the perfect tool for quickly summarizing lectures and for reviewing past recordings in a fast, and efficient manner.
How we built it
We used Python + Flask for the backend, serving a ReactJS frontend. For the audio transcription and processing, we utilized AssemblyAI's API. We started off with our concept design, planned out what features we thought were most useful then began building our MVP. After that, we kept building on as much functionality as possible before the deadline. We intend on adding more functionality after the hackathon!
Challenges we ran into
We faced issues dealing with logistics of the program, with hours of videos being processed and saved for replay. Also the issue of the paid model of AssemblyAI, and how we can make users accountable for their usage. One issue for testing was how to simulate processing delays and a realistic queueing without having to send videos to be reprocessed each run; to solve that problem we created a caching system for prior API calls to spoof processing of videos locally for debugging.
Accomplishments that we're proud of
Projects with language synthesis are always difficult, due to the complexity of the task. Thankfully, with the power of AssemblyAI we were able to get consistently accurate and helpful summaries, and useful chunking of video lectures, not to mention accurate transcriptions. Being able to make an easy-to-use website with a helpful functionality is incredible; especially the week before midterms!
What we learned
We got a chance to use AssemblyAI for the first time, and also finding out how to properly connect backend Python with a React frontend. We also worked on design together as a team, figuring out what makes a website intuitive and useful. One very practical problem we addressed was different logistics for a mainstream app, such as privacy, efficiency, and scalability.
What's next for Recap Bot
We intend to add users, so people are able to save their API keys to their accounts. We also want to be able to save past videos so that you can easily jump between other lectures for a long term study tool. Other plans include searchable keywords, to quickly sift through weeks of lectures easily, being able to share lecture summaries, and course outlines and at-a-glance reviews.
Domain.com name: reactbot.tech
Log in or sign up for Devpost to join the conversation.