MindBuddy

Inspiration

Our journey with MindBuddy began with a simple, shared observation: in a world filled with digital communication, true, empathetic listening is becoming increasingly rare. We saw friends and peers struggling with daily stresses, hesitant to speak up due to the fear of judgment (stigma), the high cost of professional help, and the simple fact that support isn't always available when you need it most. This led to our core inspiration: What if we could use the power of AI to create a truly safe, accessible, and non-judgmental space for anyone to talk? We realized that true empathy requires understanding more than just words. It requires understanding the emotion behind the words. This "aha!" moment drove us to build a companion that could analyze not just text, but the user's actual tone of voice, directly addressing the UN's Sustainable Development Goal 3.4 for promoting mental well-being.

What it does

MindBuddy is a comprehensive, multi-feature AI wellness companion. It's not a single tool, but a holistic platform designed to support a user's entire wellness journey, creating a safe space for vulnerability and growth. • Core AI Chat: At its heart is a multi-modal conversational AI. It accepts both voice and text input and uses a sophisticated pipeline of three machine learning models to provide uniquely empathetic and contextual responses. • Wellness Quizzes: To empower users with self-insight, MindBuddy includes an adaptive quiz module. Users can take quizzes on topics like "Stress Management" or "Sleep Quality" and track their progress over time, turning abstract feelings into tangible insights. • Daily Sanctuary & Activities Hub: To build positive daily habits, the platform includes a "Daily Sanctuary" with interactive, guided mindfulness exercises. Each day offers a new activity, like box breathing or grounding techniques, making mindfulness an accessible daily practice. • Personalized, Persistent Memory: Crucially, every interaction is logged to a secure, user-specific dashboard, managed by our Django backend. This allows MindBuddy to remember each user, ensuring privacy and personal ownership of their wellness journey.

How we built it

We engineered MindBuddy with a full-stack philosophy, ensuring a scalable and robust architecture. Our process was iterative, starting with rapid prototyping and evolving into a polished, deployed application. • Frontend: We chose Streamlit for its power in creating beautiful, interactive, and data-centric user interfaces with Python. It allowed us to quickly build and refine the user experience. • Backend: A robust Django application handles all user data management, session logging, and provides a scalable foundation for future features. We used PostgreSQL as our database to ensure data integrity and performance. • AI & Machine Learning Core: Our AI pipeline is the heart of the project: o Speech-to-Text: We integrated OpenAI's Whisper model (via faster-whisper), enhanced with a VAD filter for state-of-the-art transcription accuracy even in noisy environments. o Emotion Analysis: We leveraged two powerful models from the Hugging Face Hub: jonatasgrosman/wav2vec2-large-xlsr-53-english for vocal emotion and bhadresh-savani/distilbert-base-uncased-emotion for text sentiment. o Language Generation: All conversational responses are generated by Llama3 via the high-speed Groq API, which was essential for creating a fluid, real-time dialogue. • Deployment: The entire platform is deployed on Hugging Face Spaces, demonstrating our ability to ship a complete, end-to-end product.

Challenges we ran into

This was an ambitious project, and our journey was filled with significant technical challenges that pushed us to learn and adapt.

The Platform Pivot: From Gradio to Streamlit: We initially began building our prototype with Gradio due to its simplicity. However, as our vision for a complex, multi-user conversational flow grew, we hit the limits of its fine-grained control over user experience and state management. We made the strategic, and difficult, decision to migrate the entire frontend to Streamlit mid-project. This required a complete rewrite of our UI code but ultimately gave us the power and flexibility needed to build the polished, user-centric application you see today.
The Unpredictability of Real-World Audio: Handling raw audio from web browsers proved to be our most persistent technical hurdle. We battled a cascade of issues, from 403 and 503 errors with external resources to low-level data length must be a multiple of... errors from the mic_recorder component. We solved this by building a resilient, multi-step audio pipeline: we made the app self-contained by downloading assets locally, and we engineered a function to correctly format raw audio bytes into a valid WAV file before passing it to our processing models.
Taming the AI: From Generic Bot to Empathetic Companion: Our initial AI responses were frustratingly repetitive. The model often ignored the user's input and defaulted to a generic, hard-coded introduction. We realized our prompt engineering was flawed. The solution was to re-architect our API calls, separating the AI's core persona (the "System" message) from the user's immediate, dynamic context (the "User" message). This seemingly small change was the breakthrough that unlocked the AI's ability to hold a truly dynamic and empathetic conversation. ## Accomplishments that we're proud of We are incredibly proud of navigating these challenges to build a fully functional, end-to-end platform. Specifically, we are proud of: • Successfully integrating three distinct AI models into a cohesive pipeline that delivers a truly unique, multi-modal user experience. • Making the tough but correct decision to pivot our frontend technology mid-hackathon to better serve our product vision. • Engineering a robust, multi-user backend with Django that provides persistent, personalized memory for every user. • Solving complex deployment challenges to ship a live, stable application that is accessible to anyone, anywhere. ## What we learned This project was an immense learning experience, teaching us lessons beyond just the code. • The UI is the Product: We learned that the user's experience is paramount, and it's worth making difficult technical changes (like switching frameworks) to get it right. • Robustness Over Features: A simple feature that works every time is infinitely better than a complex one that fails intermittently. This lesson, learned through our audio struggles, is one we will carry forward. • The Art of Prompt Engineering: We gained a deep appreciation for how to "speak" to large language models. The structure of the prompt is just as important as the content. ## What's next for MindBuddy The potential for MindBuddy is vast. Our immediate next steps would be to: • Expand the Library of Activities: Add more guided exercises and meditations to the Daily Sanctuary. • Implement Proactive Goal Setting: Allow the AI to suggest personalized weekly goals based on quiz results and chat history. • Long-term Vision: We see the core emotion-analysis engine as a valuable asset that could be integrated into other platforms, such as telehealth services to help doctors understand patient sentiment, or in educational tools to gauge student engagement. Our ultimate goal is to continue leveraging technology to make empathetic support a universal reality.