A web app to track and increase audience engagement in real time while delivering stage talks. Sierra is here to help you with your presentations, lectures, TED talks and more importantly Online courses. Here's what it does: It detects the moods of the audience as you talk your way through the presentation. How? We analyze the sentiment from your speech in real-time. A graph is plotted for the sentiment values (positive score for positive sentiments, 0 for neutral and negative score for negative sentiments). This is done using the Microsoft Cognitive Services APIs. The Speech to text API converts your presentation speak into text in real time and displays on the screen as you speak. The Text Analytics API calculates the sentiment value for every word in your presentation speech. The score obtained is thereby normalized into a scale -100 to 100. Now for the emotion detection of the audiences. Live video is captured via the webcam and frame-by-frame extraction is carried out using opencv2. Microsoft Cognitive Science Face API and Emotion API are used to identify the emotion of the audience. The emotions can be one of the 7 pre-defined emotions: Happiness, Sadness, Anger, Surprise, Contempt, Fear and Neutral. Each emotion is given a unique score and the cumulative score is calculated for a single image. Upto 64 faces can be recognized from a single image. A parallel graph is plotted based on the values of the emotions received.

Thus, the presenter can identify in near time that his talks are not generating the required amount of excitement in the audience and can talk real time decisions to change that. Dips in audience interest levels and the corresponding speech and image snapshot can be provided along with the timestamp. This will help the presenter to better understand his audience and improve upon.

Built With

  • javascript
  • python
  • html
  • css
  • azure
  • microsoft-speech-to-text-api
  • microsoft-text-analytics-api
  • microsoft-emotions-api
  • microsoft-image-search-api
  • microsoft-cognitive-science-api
  • microsoft-face-api
Share this project:
×

Updates