PitchPerfect

Inspiration

All of our groups members are actively looking for summer internships for the first time. We feel like interviews are the hardest part of the entire process, partially because it is difficult to practice for them and get useful feedback. We aimed to solve this problem by creating a application that can be used by fellow students in our position to practice for and get better at answering interview questions.

What it does

Currently, our application prompts the user for what job title they are looking for. Additionally, they can provide a job description from a job posting they are applying too. We then take this information and make use of artificial intelligence to generate 3 interview questions for the user. The user then records themselves answering these questions. Once the questions are all answered, they are submitted for feedback. We take these videos, extract the response the user made using speech-to-text software, and then use this text, which is configured to also pick up filler words used, to an artificial intelligence model to analyze the response. The feedback, question, and response are then provided to the user to allow them to use the constructive criticism to get better moving forward.

How we built it

We used react for the front-end and flask for the backend API. We are using GCP speech-to-text to transcribe the videos, and are using openai's API for all the artificial intelligence question generation as well as response analysis.

Challenges we ran into

Sending the recording data from the front end to the back end proved a difficult challenge. As of the writing of this project story, we still have failed to get it to work correctly. This is very disappointing since it is the main part that ties together all the different functionality of our application. If we did not hit this road block, I am confident that we would have been able to properly host the application using GCP as well which is also disappointing.

Accomplishments that we're proud of

The overall functionality, while it doesn't work all together, the individual parts work well and do make us all very proud. The use of multiple different technologies to create a project of this scale in just a couple days is something we all take pride in.

What we learned

We learned that the planning phase, especially with such a limited amount of time, is very difficult. While we originally though learning how to use multiple different types of technologies together would be the hardest part, we found that seemingly simple things, such as communicating videos from our front end to our backend was actually the most time consuming and hardest to troubleshoot. Altogether, I think we have learned a lot about our potential and ability to work as a team. We all had a lot of fun this weekend.

What's next for PitchPerfect

During the inception of the idea that lead to PitchPerfect, we originally envisioned the application to analyze additional features of the video, such as body language and speech cadence. We also acknowledge that for this application to reach its full potential, it would need to be hosted somewhere, preferably the cloud, where it can scale with demand and be more available.