ProfessorAI

Your personal professor whenever you need it.

Comment 3

Inspiration

With COVID-19 forcing schooling to go virtual, learning and teaching has become more difficult than ever. Teachers have been forced to adapt all their lesson plans to a new format and students have had to encounter the issue of staying motivated and attentive. Especially with the internet just a click away, it has become even easier to lose focus. People who may need extra help due to learning disabilities and or disorders such as ADHD have it even worse, without their teachers by their sides, they may fall behind in their studies. Our team has felt this change personally, which is why we created this web application to help combat that and get people the extra help they need to stay caught up in school.

What it does

ProfessorAI contains a multitude of functions designed to facilitate online learning. Users can upload either a video file, or paste a link from YouTube. Then, a transcript of the audio is returned, along with a summary of the data. This includes key words and main topics, Wikipedia page links, current events, and recommended YouTube videos. From here, users can either read the summary report on the website, or download it as a pdf for their personal studying. Additionally, users may "ask the professor" a question, and get a quick short answer for themselves.

How we built it

We started with a NLP speech-to-text API and used it to convert audio files to a written transcript. This was initially the Google Speech-to-Text API, but after more research, we found TextRazor, which produced much more accurate results. After expanding upon its functions, we could generate an analysis report containing transcript, summary, and keywords. From there, we used the Wikipedia API to generate links based on the keywords. To provide more information, we also created a system to search recommended YouTube videos based on a search query, which used the YouTube-Data API. After realizing users would likely be uploading lecture recordings, which are video files, we built a function to convert mp4 video to mp3 audio using FFmpeg software. In order to make our resource available to a broader audience, we decided to write a script for converting YouTube videos to summaries. Using the YouTube-Data API, we pulled the auto-generated captions from videos, then called the same analysis report from the audio file version. Our next step was to set up a server using express.js to handle the HTTP requests and link our JavaScript files and html together. This used jQuery Ajax calls, along with the request() module of Node.js. Finally, we built the front-end using HTML, Bootstrap, and CSS, and linked our site to a domain.

Challenges we ran into

The biggest challenges we ran into involved setting up the server using express.js and handling HTTP requests. We wanted to use an HTML form to pass a mp4 file to the server. But, the default HTML form action request refreshed the page, losing all user inputs. Then, we tried the FormData module from Node.js, which couldn't pass an mp4 file due to size constraints. Next, we tried jQuery Ajax calls, but due to Chrome's privacy protection, the file path of any uploaded files was hidden. Our solution was to use the default HTML form request with a target set to an iFrame in order to stop the page from refreshing. Furthermore, we had issues with the execution order of functions(). When a request was made to the server, the code would move on instead of waiting for a response. In order to fix this, we used callback functions and Promises, which took a while to learn. Another major challenge was the Github and VSCode setup. Since we used Node.js extensively, we had a huge folder of node-modules, which would result in conflicts every time we tried to push and merge branches. This created a lot of setbacks and errors, which we eventually fixed by working on a LiveShare.

Accomplishments that we're proud of

ProfessorAI is a well-developed system with a proper HTML front-end and server back-end. It uses a variety of API calls, and is extremely capable. The website, built using Bootstrap, was beautifully designed with special fonts and animations. Additionally, ProfessorAI is very useful. It is capable of generating key topics, a summary of those topics, Wikipedia links, and YouTube videos. It's applications are not limited to school work. For example, instead of watching a YouTube tutorial on converting mp4 to mp3, we used our own function to generate a transcript and read that instead. We were very proud to have overcome our challenges to make a successful and appealing web application.

What we learned

This was our third hackathon, and it was the first time we used Natural Language Processing. Therefore, we had to learn everything from scratch. We did extensive research on NLP on what it actually is and what APIs to use. We learned a lot more about creating web applications, especially when it came to connecting the front-end and back-end. We had a lot of difficulties with the middleware and how to handle large files along with asynchronous javascript, but we eventually learned and figured it out.

What's next for ProfessorAI

In the future, we can increase the range of inputs that can be entered into ProfessorAI. Currently only mp4 files mp3 files, plain text, and YouTube videos are allowed; we can extend this to other file types. Additionally, we can expand on the functions that ProfessorAI is capable of, such as returning entity-sentiment, and translating into different languages.

Share this project:

Updates