Inspiration
We were inspired by helping presenters ensure that their audiences are actively engaged and listening. As presenters, it is pivotal to be adaptable to audience reactions to ensure that the information is coming across well. However, oftentimes it is difficult to multitask on stage, especially presenting infront of others is already a daunting task. Therefore, we created an app that does the hard work for you, informing presenters of live recommendations they could say to increase audience engagement.
What it does
CrowdCue uses AI facial recognition to determine the emotions of the audience. Using this data and image to text processing, we were able to incorporate the Gemini API to allow the application to provide the presenter with live/ automated insights and recommendations to improve audience engagement levels.
How we built it
We built a camera-facing side, a user-facing side, and an image-to-text extraction side.
We also implemented various existing models, but adjusted the code to fit our application's needs.
Challenges we ran into
It wasn't easy to fine-tune the model to fit our desired emotions. Additionally, since we worked on separate git branches that all required components from the other branches, combining the branches was difficult.
Accomplishments that we're proud of
We are proud to have created an application that allowed us to explore methods that we have never done before and topics that interest us. We are also proud to create an application that is applicable for us to use in future presentations.
What we learned
We learned how to incorporate live video analysis and machine learning to classify human emotions through facial expressions. We also applied (OCR) Optical Character Recognition, computer vision, and natural language processing.
What's next for CrowdCue
To improve insights and recommendations, the next feature we could add would be audio integration, pulling audio data to curate better suggested actions the presenter could take such as slowing down or speeding up.
Log in or sign up for Devpost to join the conversation.