AI Communication Assessment Platform
Inspiration
The inspiration came from observing how critical communication skills are in academics, interviews, and professional life, yet many people lack proper feedback to improve. I wanted to create a tool that not only evaluates but also guides users toward better speaking habits in a personalized manner.
What I Learned
Through this project, I learned how to integrate machine learning models, NLP, and computer vision into a seamless web platform. I also gained hands-on experience with speech analysis, grammar evaluation, and body language detection, along with deploying AI systems in a practical, user-friendly way.
How I Built It
The system was built as a Flask-based web application. The frontend uses HTML, CSS, and JavaScript to capture video/audio from the browser. The backend processes recordings using different AI utilities:
- Grammar checking via a language tool
- Speech-to-text and speaking rate estimation using ASR pipelines
- Pronunciation, fluency, and pauses analyzed from audio features
- Posture and eye contact assessment using computer vision on video frames
Finally, all metrics are combined into a comprehensive communication score with detailed feedback.
Challenges
Some major challenges included ensuring real-time video/audio recording in the browser, handling file conversion between formats (WebM → WAV/AVI via FFmpeg), and synchronizing multiple analysis modules into a single pipeline. Another hurdle was balancing accuracy vs. runtime performance, since feedback had to be both meaningful and efficient.
Built With
- ffmeg
- html
- javascript
- jspdf
- languagetool
- numpy
- opencv
- pandas
- python
- tensorflow-lite
- webrtc
Log in or sign up for Devpost to join the conversation.