MoodSync

💡 Inspiration

My colleague and I wanted to explore how technology could support mental health awareness. Many people struggle with emotions that aren’t always visible — and often, signs of stress, anxiety, or sadness go unnoticed until they become serious. Hence, we developed a program to better understand a person's sentiment by analyzing both textual and facial cues.

⚙️ What it does MoodSync analyzes human emotion in real time using two AI models:

• Facial Emotion Recognition (FER) for detecting expressions through the webcam
• Text Sentiment Analysis using a pre-trained Transformer (DistilBERT) model from Hugging Face
• It then fuses both results into a single Mood Score, visualizing the overall emotional state in an intuitive dashboard. The goal is to create AI that not only sees and reads, but also understands emotions.

🛠️ How we built it

• Used Python + Flask to build a local web dashboard for real-time emotion analysis.
• Integrated FER (Facial Emotion Recognition) for detecting facial expressions via the webcam.
• Integrated Hugging Face Transformers (DistilBERT) for text-based sentiment scoring.
• Combined both models’ outputs to generate a unified MoodSync Score.
• Used HTML/CSS and JavaScript for front-end visualization and live camera feed display.

🚧 Challenges we ran into

• Setting up the camera access and FER model with Flask caused compatibility issues on some systems, and we had to change the camera configurations to ensure compatibility and a better real-time feed.
• Integrating APIs smoothly while maintaining performance and readability
• Synchronizing the real-time visual and text models was tricky, especially managing latency between webcam frames and text input.

🏆 Accomplishments that we're proud of and what we learned

• Successfully integrated two different AI modalities (vision + language) in one system.
• Learned how to use transformer-based models (like DistilBERT) for sentiment analysis, and how facial emotion recognition APIs detect emotions using CNNs.
• Designed an interactive UI that helps visualize emotion beyond just raw data.

🚀 What's next for MoodSync

• Integrate speech-to-text with the help of a microphone
• Deploy MoodSync as a browser extension or a mobile app for platforms like Zoom or Discord
• Deploy it as a therapy product after adding more features

Built With

• html
• python
• pytorch