Inspiration
In a fast-paced digital world, people often suppress or lose touch with their daily emotional states. Most self-tracking tools require tedious manual journaling or static surveys that fail to capture real-time nuance. We wanted to build a seamless, empathetic mirror—an intelligent space where users can simply speak and look into the camera to receive an accurate, multi-modal reflection of their well-being.
What it does
MoodMirror analyzes real-time video and audio input to assess emotional health and mood states: Facial & Vocal Sentiment Analysis: Detects micro-expressions, facial cues, tone, pitch, and speech cadence. Intelligent Mood Scoring: Aggregates multi-modal data into an intuitive wellness score. Personalized Recommendations: Delivers tailored interventions, such as guided breathing exercises, ambient soundscapes, curated playlists, or mindful reflection prompts. Progress Tracking: Logs emotional trends over time to help users spot burnout patterns and celebrate positive shifts.
How we built it
- Frontend: Built with Next.js, React, and Tailwind CSS for a clean, responsive, and accessible user experience.
- AI & Processing: Integrated computer vision models (face mesh / expression recognition) alongside audio processing pipelines for acoustic feature extraction and speech sentiment analysis.
- Backend & Storage: Powered by Python/Node.js microservices and Firebase/Firestore for secure real-time data sync and user history storage.
Challenges we ran into
- Multi-Modal Synchronization: Merging continuous video frame analysis with episodic audio streaming in real time without lag.
- Lighting & Audio Variability: Handling noisy backgrounds, poor lighting, and varying camera angles while maintaining detection accuracy.
- Latency & Performance: Optimizing model inference to run smoothly in the browser without high computational overhead.
Accomplishments that we're proud of
- Creating a low-friction, natural interaction where users do not need to fill out tedious forms to understand their mood.
- Successfully combining audio and visual metrics into a single, cohesive mood index.
- Designing a calming, privacy-first interface that makes emotional check-ins feel safe and empowering.
What we learned
- Ethical AI & Nuance: Human emotions are deeply contextual; relying on a single data point (just voice or just face) leads to false reads, making multi-modal fusion essential.
- Client-Side Optimization: Techniques for offloading and batching frame analysis to preserve frame rate and battery life on mobile devices.
- User-Centric Design: Mental health tools require an empathetic, non-judgmental tone in both copy and UI design.
What's next for MoodMirror
- Wearable Integration: Syncing with smartwatches to incorporate heart rate variability (HRV) and sleep data.
- AI Copilot & Conversational Journaling: An interactive voice companion for guided conversational check-ins.
- Offline On-Device Inference: Transitioning to fully local WebAssembly/WebGPU models to guarantee maximum user privacy.
Log in or sign up for Devpost to join the conversation.