Tells the presenter whether the audience is paying attention to your presentation.
What it does
Measure the attention rate and emotional response of audience and predict the success rate of the speech
How we built it
Attention Rate: Our software measures people’s attention rate by doing vector analysis on people’s head. Using the Microsoft Azure Face API, our software determine the position of each audience’s faces and the way their face is facing. By performing vector analysis, we can know that if the person is facing the presenter, which means that they are paying attention to the presentation. Otherwise, if the person is looking at other directions, like if they are talking to others or typing on their phone, we can be sure that they are not paying attention. By averaging the each person’s attention rate, we can update a live analysis on the group attentaion rate, which is shown in the top right part of the screen
Emotional Analysis What if you want to know people’s emotional reaction to your speech? By using the Microsoft Emotion API, we can figure out each individual’s emotional reaction to your speech at any time you want. By simply clicking the emotion tab, and then click on the face that you want to discover, spotlight will give you the three most significant emotion. You can always look at the emotional reaction of the current time by clicking the emotion tab again. Now you see people’s current reaction.
Success Prediction What if you want to know if your audience is truly amazed by your presentation, you can always predict the success rate of your speech. Based on the history of group success rate and people’s emotional reaction, we use decision tree to predict whether your speech is going to be successful based on the two hundreds preloaded data samples in our machine learning model. You can actually see the decision tree growing in this part of the software.
Challenges we ran into
API run time is too long : pre-processing of data, multi-threading
Accomplishments that we're proud of
Successfully measuring the success rate, and the emotional response
What we learned
opencv development, multi-threading synchronization, vector analysis based on API result, MVC with tkinter, MicroSoft FaceAI, Microsoft EmotionAI
What's next for Spotlight
More accurate measurement