Breathe

Connect, Share, Thrive

Comment

Inspiration

In a world where mental health remains shrouded in stigma, our vision is to create a groundbreaking solution that transcends barriers and brings mental health support to every corner of the globe. Recognizing the need for accessible, personalized, and stigma-free mental health care, we have embarked on a mission to leverage the power of artificial intelligence. Our inspiration stems from the desire to break down the walls that hinder individuals from seeking help, providing a seamless and innovative platform that not only assesses mental well-being but also fosters meaningful connections and professional support.

We tried to build in the most authentic and secure way to ensure the privacy of the user while using the application. Our motto is simple

Connect - with like-minded people facing similar issues mapped by embeddings, because of all the time we have opened up to strangers we just met because we are facing the same adversity. We also map to professionals based on their specialization

Share - your stories with each other to help through tough times

Thrive - own your mental health and carry on with confidence!

What it does

Our revolutionary application combines advanced technologies to comprehensively analyze a mental health patient's condition. By integrating data from wearable smartwatches, mobile phones, and voice emotion analysis, our system generates a real-time assessment of the user's mood, mental state, and emotional well-being. The culmination of this analysis results in a unique Mental Health Index, a personalized metric that serves as a guiding beacon for the user's mental health journey.

The application goes beyond mere assessment – it takes a proactive approach to support. If the Mental Health Index falls below a predetermined threshold, the user is seamlessly connected with a peer experiencing similar challenges. This peer-to-peer connection serves as a source of empathy, understanding, and shared experiences, breaking down the isolation that often accompanies mental health struggles.

For cases requiring professional intervention, the application seamlessly facilitates a connection with mental health professionals or doctors. It goes the extra mile by providing these professionals with a live report during video consultations. This report not only includes a comprehensive overview of the patient's mental health condition but also extracts keywords from their voice interactions. These keywords serve as tags, aiding in future identification and streamlining health record management for improved continuity of care.

Our vision extends beyond the urban landscape, aiming to eliminate disparities in mental health access. By leveraging AI, we aspire to create intelligent applications that dismantle social barriers, making mental health services as effective and accessible in remote and rural areas as they are in metropolitan regions. This project represents not just a technological leap but a commitment to fostering mental health equity worldwide.

How we built it

Our innovative mental health support application was meticulously crafted using a combination of cutting-edge technologies and robust frameworks. The development process was driven by our commitment to seamlessly integrate various components to deliver a comprehensive and user-friendly solution.

1. React Native on Convex for Application Development: We chose React Native on Convex as our development stack to ensure a cross-platform, responsive, and efficient user experience. Leveraging the power of React Native allowed us to streamline the development process, ensuring that our application performs seamlessly on both iOS and Android platforms.

2. Terra API for Health Metrics: Integration with the Terra API played a pivotal role in collecting health metrics from devices such as smartwatches and mobile phones. This API facilitated the retrieval of real-time data, including vital signs and activity levels, providing a holistic view of the user's physical well-being.

3. Monster API Speech-to-Text Conversion: To harness the valuable insights embedded in users' voices, we utilized Monster API's Speech-to-Text conversion. This integration enabled our application to analyze spoken words and emotions, enriching our dataset for a more comprehensive assessment of the user's mental state.

4. ChromDB for Word Embeddings: ChromDB became an integral part of our technology stack for obtaining word embeddings from user interactions. By leveraging ChromDB, we transformed spoken and written words into vector representations, enhancing the depth of our natural language processing capabilities.

5. Together.ai's LLM Model (Mistral-7B-Instruct-v0.1): At the core of our mental health assessment lies the Mistral-7B-Instruct-v0.1 LLM (Large Language Model) from Together.ai. This state-of-the-art model, designed to understand and assess textual data, provided a sophisticated framework for evaluating the patient's mental health condition. Its ability to comprehend nuances in language, context, and emotions greatly enhanced the accuracy and depth of our assessments.

In essence, our development process was driven by a commitment to integrating the best technologies available, ensuring a seamless, reliable, and clinically rigorous solution for mental health support. The synergy of React Native, Monster API, Google Cloud, ChromDB, and Together.ai's LLM model enabled us to create an application that not only breaks down barriers but also sets a new standard for AI-driven mental health care.

Challenges we ran into

Embarking on our first Hackathon presented a unique set of challenges, compounded by the integration of cutting-edge technologies that our team was using for the first time. Here are some of the key challenges we encountered:

  1. Learning Curve with React Native and Convex: Given that it was our team's first experience with React Native and Convex, we faced a learning curve in mastering these technologies. Adapting to the nuances of cross-platform mobile application development required dedicated time and effort, impacting our initial development pace.

  2. Integration Complexity with Monster API: Integrating the Monster API to fetch health metrics from devices posed a challenge due to its complexity and the team's initial unfamiliarity. Configuring the API calls, handling data responses, and ensuring seamless communication with various devices required thorough understanding and experimentation.

  3. Speech-to-Text Integration Challenges: Incorporating Google Cloud's Speech-to-Text conversion for voice analysis presented challenges in terms of understanding the nuances of the API, configuring real-time transcription, and handling potential issues such as background noise and varying speech patterns.

  4. Navigating ChromDB for Word Embeddings: Utilizing ChromDB for word embeddings was a novel experience for the team. The process of converting spoken and written words into vector representations required overcoming challenges related to data preprocessing, ensuring compatibility, and optimizing for efficient integration.

  5. Model Understanding and Tuning for Mistral-7B-Instruct-v0.1: Leveraging Together.ai's Mistral-7B-Instruct-v0.1 LLM model was a groundbreaking yet challenging aspect of our project. Understanding its architecture, fine-tuning parameters, and optimizing its performance for our specific use case demanded a deep dive into the world of large language models, which was a novel territory for the team.

  6. Scheduling appointments with the mental health professionals nearby using fetch.ai: Fetch.ai's agent is used to map patients with nearby doctors and automatically schedule an appointment and save it in the doctor's calender.

  7. Coordination and Time Management: Participating in a Hackathon for the first time required effective team coordination and time management. Balancing the learning curve of new technologies with the need to deliver a functional prototype within the limited timeframe was an ongoing challenge.

Despite these challenges, our team embraced the opportunity for growth, learning, and collaboration. The experience served as a valuable lesson in overcoming obstacles, fostering resilience, and solidifying our commitment to creating a groundbreaking solution in the mental health space.

Accomplishments that we're proud of

We are proud of our idea and commitment - the amount of energy and passion that we put into this project to solve a real-world problem was something special and close to our hearts. The accomplishment that we are most proud about is the impact that this product could have on elevating the efficiency of the healthcare system especially in the mental health space.

What we learned

We gained a deeper understanding of mobile application development, data integration from diverse sources, and the intricacies of large language models. Learning to manage our time efficiently and coordinate efforts within the constraints of a Hackathon timeframe became a skill set we honed.

What's next for Breathe

We had a lot of plans when we came here, but only implemented a subset. We have a highly positive vision for Breathe

  1. Live emotion-tracking during calls using a meter for the therapist
  2. Mood tracker dashboard on the profile page for the user to visualize their mental health journey based on embedding-generated data
  3. A real-time AR-VR interaction model for therapy to enable a personalized experience closer to human interaction

Built With

Share this project:

Updates