Raise your hands if you have scrambled through your lecture notes/videos at the last moment before an exam. Yes! All of us have done that! No matter how early we start preparing for the exams, last-minute preparations always help. Browsing through Piazza posts, lecture slides, and videos to find a concept during the last minute is time-consuming and stressful. We always felt we could save an ample amount of time if someone could just point us directly to the resources we need to clarify our doubts. This is where the idea for Spot.It was born.
What it does
Spot.It gives us a very simple and intuitive interface to search through the lecture videos to find exact points in the videos where the professor explains the topic, gets you the top results from the internet and a curated list of short (< 5 mins long) videos from Youtube, all in one place and with just one search. So, we won't have to spend time searching through our lectures or online references to clarify our doubts.
How I built it
We used Angular 9 and NestJs (both typescript based frameworks) to build frontend and backend of the application. We also used Facebook's Wit.ai to perform Natural Language Understanding on user's query and used the results from the intent processor to perform queries on the data in our system
Challenges I ran into
The main challenge was to fetch the right results that save student's time while preparing for the exam. There were a lot of great experiences during the roadmap of the product.
- We needed a reliable and highly accurate source of search results from the internet and there aren't any free API for that. Scrapping Google would have end up getting the backend server's IP blocked. We tried Duckduckgo and Rapid API as well, but either the APIs weren't free or the results weren't as accurate as needed. Thanks to Microsoft, we ended up fetching results from Bing!
- Transcribing Youtube videos was another good challenge. Started with a python script, worked well in the start till getting stuck at Youtube's bot captcha. We tried a third party API which resulted in getting IP blocked (We wonder if they scrapped results :P). Finally, we settled with another npm package that uses a legitimate way to transcribe Youtube.
- You can't just search of the Youtube transcriptions for the user's query as videos are of varying duration with multiple occurrences of user's query, resulting in getting numerous results of annotations timestamps which were only a few seconds away. We added functionalities which help the system decide whether the user should watch the full video in case of shorter videos, and also to get only the most confident spots in case of longer videos, instead of sending all the possible spots.
- Last but not least, getting time out for the project while working a full-time job!!
Accomplishments that I'm proud of
- It works! This project is a proof of concept for what we envisioned about software doing such kind of tasks. We are pretty sure all the students out there would find this helpful.
- Angular changed a lot since we last worked on it so it was a great brush up on that.
- Last but not least, Wit.ai is awesome Its simple, intuitive, and no API calls limit. The best part the interface to train for the utterances. Very cool
What I learned
- Sometimes things don't go the way you expect them to. The second Youtube transcription approach started failing just a few days before submission
- Wit.ai is a great tool for building chatbot like experiences. We weren't aware of it until we found about this hackathon. Mostly worked with Google's Dialogflow.
What's next for Spot.It - Get Faster Exam Prep
- Alot ! This version of Spot.it is just a proof of concept running on free servers and using some free resources available online. Future roadmap includes adding the capability of searching through lecture slides, connecting with student discussion platforms like Piazza or Facebook Groups to get more curated and accurate results to make last-minute preparations even faster.
- Adding support for uploading videos from other platforms.
- Adding support for user signup
- Adding support for voice-based search using speech to text
- Last but not least, getting better infrastructure resources to scale-out