Whispers of the Heart

Analysis of the jpurnal image input
Analysis of the journal text input

Inspiration

There's a prevalent misconception that AI can replace professional therapists, which undermines the nuanced understanding and expertise therapists provide. We were inspired to develop a GenAI App that leverages AI to complement therapists' work rather than replace it. Unlike virtual AI therapy apps that lack professional guidance, our goal was to create a tool that assists therapists in understanding their clients better.

What it does

Our app serves as a supportive companion in the journey of self-discovery and personal development. It provides users with a safe and judgment-free space to express themselves openly without fear of scrutiny. Unlike traditional therapy apps, our app does not aim to replace the therapeutic relationship but rather enhances it by serving as a discreet observer. The app respects the domain knowledge and expertise of therapists and psychoanalysts. It acts as a wallflower, preserving the core purpose of journaling without making any judgments or diagnoses. By leveraging generative AI, the app analyzes users' journal entries, poems, and artwork to interpret their thoughts, feelings, and moods. It identifies and highlights key emotions and thought patterns, assigning confidence and intensity scores to each mental state. The app tracks the emotions and thoughts of individuals, providing therapists with observations and insights. The analysis generated by the app includes emotions, possible thought patterns, mental well-being scores, a summary of the day, and an explanation of the reasoning process.

How we built it

Development: We built the app from scratch using Python and integrated Gemini APIs like gemini-1.5-pro-latest and gemini-pro-vision.
App Views: The app consists of two views: the journal writer view and the therapist's view. Gemini facilitates analysis for the therapists while ensuring the privacy of journal writers. The app is designed not to undermine the purpose of journaling but to provide analysis to therapists.

Challenges we ran into

Inconsistent AI Responses: Generative AI's inconsistency in generating content posed a challenge, requiring careful refinement of prompts and instructions.
Image Interpretation: Vision APIs provided inconsistent interpretations of images, especially when run multiple times.
API Limitations: Limited usage and API calls for the latest models, such as gemini-1.5-pro-latest, posed constraints on scalability and development.

Accomplishments that we're proud of

Responsible AI Use: We prioritized the responsible use of AI by ensuring that our app complements professional therapists rather than replacing them. This approach aligns with our belief in the importance of human expertise in therapy.
Innovative Use of AI: We have pushed the boundaries of AI apps by leveraging advanced APIs like Google's Gemini API to enhance the analysis of journal entries, poems, and artwork. This innovative use of AI has enabled us to provide therapists with deeper insights into their clients' experiences.
Empowering Therapists: We have empowered therapists to provide more personalized and effective therapeutic interventions by offering them a nuanced analysis of their clients' input. This enables therapists to make informed decisions and tailor their interventions to better support their clients.
Privacy and Autonomy: Our app respects the privacy and autonomy of journal writers by acting as a discreet observer. We have ensured that the app does not output anything to the journal writer, preserving the core purpose of journaling as a safe space for self-expression.

What we learned

Leveraging Advanced APIs: We explored Google's Gemini API, especially gemini-1.5-pro-latest and gemini-pro-vision, to push the boundaries of AI apps and enhance the analysis of journal entries, poems, and artwork.
Prompting Techniques: We experimented with different prompting techniques to elicit appropriate responses from the AI, improving the accuracy of the analysis.
Responsible Use of AI: We learned the importance of using AI responsibly to complement professional therapists rather than replacing them. We prompted the AI to articulate its thought process, ensuring transparency and trust.
Accountability: After the AI completes the analysis, we asked it to explain its reasoning step by step, including examples and assumptions.

What's next for Whispers of the Heart - A daily reflection journal

The idea is to develop fully fleged application based of this app idea. The fully built application will have:

Full capabilities Google's Gemini API to empower GenAI apps to foster innovation in the area of therapy and pyschoanalysis.
End-to-end encryption for the journal and analysis.
Time series analysis of different mental states using pyschoanalysis domain knowledge and expertise.
Better interface for both journal writer and the therapists.
Incorporation of "honest but curious" policy between the therapists and writer to promote privacy and transparency.