Inspiration
We wanted to help detect mental health issues through an app that people would feel comfortable using in their daily lives. Mental health challenges are more common than we think, and they affect everyone. By making self-reflection and mental health monitoring more accessible, we hope to provide early insights that encourage users to seek help when needed.
What it does
Journal AI is a voice-based journal assistant that talks with you about any situation or feeling you'd like to share. It transcribes your voice, analyzes your emotional tone and language, and evaluates your mental state using parameters such as mood, energy level, cognitive patterns, and crisis indicators. It then offers personalized insights and recommendations based on your input.
How we built it
-We built Journal AI using: -React for the front-end interface, with a focus on accessibility and clean design. -Flask (Python) as the backend server to handle API requests and process data. -OpenAI's Whisper for real-time speech-to-text transcription. -OpenAI's language model to generate reflective responses and assess emotional and cognitive patterns. -WebRTC and Web Audio API for real-time audio streaming and interactive visualizations, including a circular waveform that reacts to voice. -Recharts to visualize emotional metrics like energy level in an intuitive way.
Challenges we ran into
- The RealTime conversation from OpenAI was a real challenge to make it work, control the output and the voice detection.
- We also had a lot of problems with the Voice-to-Text
Accomplishments that we're proud of
We are proud of being able to finish the project that we intended to do. We developed the mental health coach that is capable of having a conversation and hyping up the user when he/she is sad.
What we learned
We have learnt to integrate OpenAI API with their different services(Whisper API, conversation and realtime API) . This last one was a real challenge because it is a new release and we had to work hard on it for it to work.
Log in or sign up for Devpost to join the conversation.