Annotating Turing Machines
As students, we oftentimes find watching hour long lectures hard to sit through and listen to in one go. It’s easy to let our minds wander and get distracted; in fact, it’s likely that we fall behind, making it more difficult to stay focused. This is what inspired the creation of ZeroIn. We built ZeroIn with the intention of stopping this cycle by providing students with digestible breakdowns of lectures, highlighting the key points and providing references to the topics being taught. After the user has uploaded their video / audio file to our website, they can find time stamps to what our algorithm found to be the main points of the lecture, along with useful links to resources that can help them explore the topic further.
In the same vein, our minds need relaxation from time to time. Concentration becomes difficult after hours of lecture, or a lack of sleep; that’s why study breaks are extremely important for productivity, and in fact can help prevent procrastination in the long run.  This is where the Study Break feature comes in. Inspired by our own passion for sports, users are able to upload a video for which they would like a highlight reel produced. Through the processing of key words from commentators, and analyzing the audience’s reaction throughout the game, we can cut out the boring parts of a video, and only retain the best highlights.
Microsoft Bing Search API was used to query relevant resources for key phrases parsed by Microsoft’s text analysis service.
Microsoft Speech to Text was used to translate audio for further processing by the Bing Search API and Text Analysis API. Since we utilized HTTP Requests against their REST API, we were restricted to the 15 second video size; a workaround was to chunk long videos (which sacrificed quality of the translation)
Microsoft Text Analysis was used to make key assumptions about a video in order to highlight the important excerpts and make smart references to relevant learning sources.
MoviePy was used to process graphics and audio inputs. Videos had to be spliced, chunked, and transformed in order to comply with some of the services Microsoft had, and to make algorithmic computation easier.
React + Redux + Laravel were used as our frontend services on heroku. Video upload was implemented on this site along with a dynamic rendering of video annotations.
Flask was used as our backend service to receive file uploads from our client, which were then processed afterward. The service stored and served videos for the client to stream.