SocialEase

Home Page
Real-Time Conversation Coach
Conversation Practice
Chat History

Inspiration

Social anxiety affects millions, making everyday conversations stressful and overwhelming. Many struggle with maintaining eye contact, reading social cues, or finding the right words in real time. We wanted to leverage AI to create an accessible tool that provides instant, personalized feedback—empowering users to practice conversations, gain confidence, and improve their social interactions in a safe, judgment-free environment.

What it does

SocialEase is an AI-powered real-time social coaching app that helps users improve their conversational skills. Using speech recognition and facial emotion analysis, the app provides instant feedback on tone, pace, word choice, and nonverbal cues like eye contact and facial expressions. Users can practice different scenarios—small talk, job interviews, or public speaking—while receiving tailored suggestions to refine their interactions.

How we built it

We used Google's Speech-to-Text for speech recognition and a custom trained ML model for facial emotion analysis. The backend is powered by Serverpod for processing real-time feedback, while the frontend is built with Flutter for cross-platform accessibility. We integrated Anthropic's Claude LLMs for conversational guidance and suggestions, ensuring smooth interaction and analysis.

Challenges we ran into

Real-time Processing: Analyzing speech and facial expressions simultaneously while keeping latency low was challenging.
Emotion Detection Accuracy: Mapping facial expressions to emotions required fine-tuning our ML model for diverse demographics.
Personalised Feedback: Designing dynamic responses that feel natural and helpful without being overwhelming.
Multi-Platform Optimisation: Ensuring smooth performance across mobile and desktop versions.

Accomplishments that we're proud of

Successfully built a real-time AI-driven feedback system that responds instantly to user interactions.
Achieved high speech and facial emotion recognition accuracy despite hardware limitations.
Created an intuitive and engaging user interface that makes practicing social skills fun and stress-free.
Developed personalised AI-driven conversation coaching, tailored to different user needs.

What we learned

Optimising AI models for real-time feedback requires balancing accuracy and speed.
User experience is crucial—clear, actionable, and non-intrusive feedback makes AI coaching more effective.
Handling diverse accents and facial expressions requires continuous model improvement.
Cross-platform frameworks like Flutter can be challenging to learn but rewarding when implemented correctly.

What's next for SocialEase

Expanded Scenario Training: Add more conversation templates like networking, dating, and debate simulations.
Voice and Body Language Coaching: Provide feedback on hand gestures, posture, and vocal modulation.
VR Integration: Create an immersive practice environment using virtual reality.
Community Support: Allow users to practice conversations with AI-driven avatars or real people in a safe, anonymous setting.
Gamification & Progress Tracking: Introduce challenges, levels, and AI-powered analytics to track improvement over time.