EloquenceAI

Inspiration

All of us in our team constantly face situations, whether in college or at work, where we need to speak clearly and confidently. Whether it’s presenting in class, sharing research findings, or articulating ideas in meetings, strong communication is key. As students conducting technical research at our universities, we often struggle to explain complex ideas in a way that’s both thorough and concise. Thus, we built Eloquence.AI to solve this real need: a tool that helps us refine our delivery and feel confident when speaking under pressure.

What Sets Us Apart

What sets Eloquence.AI apart is its multimodal intelligence—it doesn't just analyze your words, but your tone, facial expressions, and body language. Unlike existing tools, we incorporate emotion detection, gesture tracking, and eye contact analysis to give users a complete view of their communication style. By combining AI-powered insight with a gamified experience, Eloquence.AI transforms the traditionally dull, anxiety-inducing task of public speaking into a motivating, actionable, and even fun journey. It’s more than feedback—it’s a personalized coach that helps you grow with every speech.

What We Learned

We explored how to combine speech-to-text models with LLMs to generate useful and personalized feedback. We also learned how to use computer vision to analyze body language, eye contact, and facial expression—turning raw video data into actionable insights. Most importantly, we learned how to tune AI systems to give feedback that feels human and helpful, not robotic.

How We Built It

We used OpenAI's Whisper to transcribe recorded speeches and GPT-4o to analyze clarity, pacing, filler words, and argument strength. We integrated computer vision tools like MediaPipe to track eye contact and gestures, giving a more complete view of a speaker’s delivery. Our frontend was built with React and Tailwind for a clean, user-friendly interface, and the backend runs on FastAPI. We also developed our own machine learning model, trained, and tested it on our own.

Classroom Integration & Gamified Learning

Eloquence.AI can be used in classrooms to make public speaking practice more interactive and fun—similar to how Kahoot transforms quizzes. Students can log in with individual accounts, complete speech challenges, and earn Elo points based on performance. A live leaderboard allows educators to track progress, encourage improvement, and spark friendly competition. This gamified system turns communication training into a dynamic experience, helping students build confidence while enjoying the process.

Challenges We Faced

One major challenge was syncing multimodal inputs, video, audio, and text, and presenting feedback in a seamless way. We also had to carefully craft prompts for GPT-4o to ensure feedback was specific and motivating. Balancing technical depth with usability under time pressure was tough, but it pushed us to make smart design decisions and prioritize the core user experience.

!(/Users/arav/Downloads/designdiagram.png)