EmoGuardian AI

AI Assistant for Mental Health Support

Inspiration

Depression manifests in many forms. It’s not just sadness; it’s the overwhelming lack of motivation to perform even the most basic daily tasks, like brushing your teeth or getting out of bed. We wanted to create a tool that not only understands these struggles but also provides a supportive, non-judgmental space to help individuals navigate their mental health journey.

What it does

EmoGuardian AI is your casual, empathetic AI companion. It engages in natural conversations, helps you journal your thoughts, and provides personalized suggestions based on your mood and needs. It remembers your previous conversations and journal entries, offering insights and habits that have helped you in the past. Additionally, if the AI detects signs of a mental health crisis, it can alert your emergency contacts to ensure your safety.

How we built it

Frontend: React + Vite for a smooth, responsive user experience.
Backend: Flask and FastAPI (Python) for robust server-side logic and API management.
Frameworks/Tools:
- Agno: For conversational AI agent that handles negative emotional response and integrates happyrobot.ai to sent crisis message to friends/family.
- Together AI: For training and deploying machine learning models.
- Happyrobot.ai: Crisis management with sms feature.
- ZenML: For building Speech Emotion detection model with ZenML training pipeline and deploy it on remote server.

Challenges we ran into

Spotify Integration: We faced difficulties integrating Spotify with Stytch to enable mood-based song recommendations. The goal was to have the AI detect the user’s mood and play music to uplift them, but technical limitations slowed progress.
Dataset Access: Finding free, high-quality datasets for text and audio-based depression and anxiety detection was challenging. Many datasets are restricted to academia, and requesting access over the weekend delayed our progress.
Third party Integration: Used ZenMl, to run and deploy our machine learning model pipeline and but faced several issues running the model on cloud services.

Accomplishments that we're proud of

-Update: After the help of zenML team we achieved the accuracy of 92% greater than the SOTA

We trained an audio emotion recognition model on the RAVDESS dataset using ZenML, achieving 75% accuracy. While the current state-of-the-art (SOTA) is 86%, we’re proud of this milestone and plan to improve the model with more diverse data sources.
We successfully built a functional prototype that integrates journaling, mood tracking, and crisis detection, all while maintaining a casual, empathetic tone.

What we learned

We gained hands-on experience with cutting-edge tools and frameworks for AI development, including ZenML for ML pipelines and Together AI for model training.
We learned the importance of seamless integration between frontend and backend systems, as well as the challenges of working with third-party APIs like Spotify.
Most importantly, we learned how to design AI systems that prioritize empathy and user-centric experiences.

What's next for EmoGuardian AI

Model Improvement: Train better models by curating multimodal datasets (text, audio, and visual) to create a comprehensive solution for mental health support.
AI Agents: Integrate multiple AI agents to assist with journaling, habit tracking, and progress monitoring, making the system more versatile and helpful.
Empathy and Personalization: Refine the AI’s conversational abilities to make it even more empathetic and tailored to individual users.
Crisis Support: Enhance the crisis detection feature to ensure it’s accurate, reliable, and sensitive to user needs.
Community Building: Explore ways to connect users with supportive communities or resources to foster long-term mental well-being.

EmoGuardian AI is just the beginning. We’re committed to creating a tool that not only supports mental health but also empowers users to take control of their well-being in a compassionate, understanding way.