Inspiration
We often found that watching long-form educational videos was quite tiring, especially when needing to reference similar sections of a video multiple times. Using our previous experience with the GPT API, we decided to use semantic searching and the API to build a project to help students in similar situations.
What it does
Our video question-answering bot uses content from a youtube video to generate brief and understandable responses for students to quickly understand long and complicated videos. Students can ask questions about the youtube video in the chat box and instantly get responses to better understand the video topic.
How we built it
We utilized the open-source OpenAI whisper model to transcribe scraped youtube videos. We then run an embedding model on both the transcription and question and then sort the transcription through the cosine similarity of both embedding vectors.
Challenges we ran into
We struggled with transcription times, switching multiple times between building the model locally and using API's, and eventually settling on a cloud environment running "locally". Integration between front-end and back-end was challenging as well.
Accomplishments that we're proud of
Creating a rest-API to integrate the front and backend.
What we learned
How to integrate well between front-end and back-end, and testing code on multiple environments before moving onto the next steps.
What's next for ReAInvent
Working with more media sources, decreasing transcription times, and adding additional features to allow for quicker parsing, and improving UI elements.
Log in or sign up for Devpost to join the conversation.