Read the room

Inspiration

We created Read the Room because we wanted to help public speakers improve their communication style. Many speakers struggle to gauge audience engagement and emotional reactions during their presentations. Some rely on visible cues like facial expressions or body language, while others get feedback only after their talk. We tried to make an AI-powered tool that could provide real-time insights, helping speakers take notes on their tone, pace, and content for better audience connection.

What it does

Upload a video to the website and it'll play it back while showing you the fluctuating emotions of the audience. This can be a great tool for people inexperienced at public speaking and can help them improve by understanding their weak-points.

How we built it

Backend Development (Flask, Python)

Integrated Azure Speech SDK for speech-to-text conversion.
Used AWS Facial Recognition to analyze facial expressions.
Processed and combined speech sentiment with facial analysis to generate overall audience mood insights.
Built a Flask API to handle data communication with the frontend.

Frontend Development (Vue.js)

Created a real-time dashboard displaying audience sentiment trends.
Designed an intuitive user interface for speakers to easily interpret feedback.
Implemented WebSockets and API calls for live updates.

Challenges we ran into

Real-time Processing: We managed to make the individual parts of the project (frame transcription, facial recognition...) work. But we couldn't make them work together to display on the frontend properly on time.

Accomplishments that we're proud of

Frame-by-frame emotional feedback: We managed to get the emotional input for a video frame by frame using the azure speech SDK and aws facial recognition API.

What we learned

Speech Processing: Implementing Azure Speech SDK to transcribe and analyze spoken words was a great learning experience. We explored different speech recognition models and learned how to handle accents, background noise, and real-time transcription challenges.
Facial Recognition & Emotion Detection: Using AWS Facial Recognition, we explored how AI can detect emotions like happiness, sadness, confusion, and engagement levels based on facial expressions. We learned the importance of lighting conditions, camera angles, and data privacy considerations.