MyInterviewPal

Magd needs more eye contact!

Inspiration

As a computer science student, interviews for internships and jobs are something that are on my mind. It is quite evident that this seems to be the case for other people as well! Surely you could do a mock interview or some whiteboard practice, but how would you gauge how well you are making eye contact, smiling, how often you are saying "uhm"? I wondered if there was an application I could build to tackle this problem.

What it does

Film yourself doing a mock interview where you pretend the interviewer is the camera. Put your video into the software and get a video output that comes with a whole slew of specs overlayed the original video you submitted. This includes your average WPM, the number of uhms you've said, the emotions your face is exhibiting, the percent of the time you make eye contact.

How I built it

I used OpenCV and TFLearn to do some head tracking/facial emotion recognition. I referenced a lot of great scripts that others before me had built. I used CMU pocketsphinx to convert the video's audio to text in order to compute metrics like number of 'uh's and WPM.

Challenges I ran into

The quality of the speech to text was not particularly good, perhaps due to the raw audio from my mic having a lot of bg noise. Also the emotion recognizer needs more training. WPM needs to be refined by removing noise in the background. As can be seen in the demo, a large amount of background noise introduces artificial increases to the words per minute.

Accomplishments that I'm proud

Quickly prototyping an application that could be very helpful for job applicants who want some quantitative data about how they interview.

First hackathon!

What I learned

OpenCV. TFLearn.

What's next for MyInterviewPal

Add more metrics so that the app is more useful (how loudly, clearly someone is peaking, how polite one is being). Do efficacy study to see how the app impacts peoples' preparation for interviews.