-
-
logo
-
home page
Inspiration
People always say -- writing something down is the best way to remember it -- and we figured typing it would be too! Transcribing lectures is nothing new, but a more interesting idea was taking that transcription, summarizing it, and then letting you type it out in a MonkeyType style browser game!
What it does
Simply upload a lecture video and "clip" the most useful time stamps with the click of a button as you watch through. Our website will summarize the most important points from those moments, and let you type it out!
How we built it
We used React for the frontend and Flask for the backend as we had a lot of python libraries we wanted to use to build this, such as ffmpeg. When the user uploads their video we display it with the React player and save the clipped timestamps in an array. This is then passed to the backend using CORS, to use ffmpeg/moviepy to chop it based on the "clipped" timestamps, and separates the audio files to be analyzed. The audio is then passed to Google Cloud's Speech-To-Text API which transcribes it as a string. We then make an API call to Cohere, passing the newly transcribed text to be summarized. This is finally sent back to the front end where it is displayed in our MonkeyType style game.
Challenges we ran into
Our first idea was a complete failure, as we simply didn't have the technical expertise to execute it. Our entire first night was spend working out how we'd do it and by 3am we realized it wasn't feasible in the time we had, with the knowledge we had. Most of us were also less experienced with web development, especially React, so we ran into a lot of trouble getting the front and backend to talk with HTTP requests. This is the biggest tech stack most of us have used before, so managing it and making sure our GitHub didn't implode was one of the biggest struggles, but we pulled through!
Accomplishments that we're proud of
Our website is functional and actually looks pretty good thanks to one VERY dedicated front end dev cooking up some cool CSS. Additionally, as we said earlier, this is the biggest tech stack we have ever used, and there were at least 3 or 4 times throughout the hackathon we considered giving up, but in the end we actually finished!
What we learned
Perseverance is key at hackathons, and there's no shame in reducing scope multiple times to make sure you can get something working to present. We also learned React from practically zero knowledge (apart from one person with mild knowledge), which meant it was also our first time trying to integrate React with Flask.
What's next for Recollect
It's possible that we'd try to recreate the app with a cleaner stack and an actually usable login system (perhaps with OAuth) that could allow you to save your previous lectures, clip data and all, and come back to them later. Additionally, there is a lot of opportunity for Quality of Life changes, such as displaying your clips at the side with AI generated titles based on their transcripts to get a better idea of what you have clipped already. Additionally, sharing your lecture summaries with others and competing for high scores (in words-per-minute, for example) could help people learn even more efficiently with competition involved.
Log in or sign up for Devpost to join the conversation.