Inspiration
Developed countries are performing incredibly well macroeconomically. However, increasingly, mental health issues are becoming increasingly prevalent. Diagnoses of depression, anxiety and other psychological illnesses are unprecedented. The need for psychological support from qualified psychologists theoretically is at an all time high, however, the supply is seemingly inadequate in Australia. Current psychology services for treatment of these psychological illnesses are for some Australians too expensive or unjustifiable during other cost of living pressures and or financial constraints. Medicare does not fully subsidise psychologists unlike GP visits.
Ideally, all mental health issues are handled by a qualified psychologist. However, for those unwilling or unable to be seen by a psychologist (which could be the vast majority of those requiring help), I wanted to leverage the power and effectiveness of newly emerging large language models, to provide some mental health assistance. These are not supposed to be a replacement, but rather assistance for those unable to reach the highest standard of care able to be provided by a psychologist.
From researching the current state of mental health care provided by chatbots, I have seen the biggest issue is the concern of hallucinations, and inappropriate advice given by the chatbots. As a result, I have decided that the primary concerns of my chatbot, are that it follows proper cognitive behavioural therapy (CBT) techniques used by psychologists and immediately refers patients with signs of suicidal ideation to professional help lines.
What it does
It is a mental health chatbot designed to imitate therapy techniques for those unable to afford or access therapy easily. It is not meant to be a replacement for professionala psychologists however, and is only for those who really cannot access a real psychologist.
How we built it
I used langchain, langgraph, streamlit, neural networks and open source LLMs. Good prompting techniques used on LLMs allow for good cognitive behavioural therapy techniques used by the LLM and suicide detection gaurdrails to redirect to professional help if needed.
Challenges we ran into
Some things I learned in doing this project are that fine tuning an open source LLM to speak like a therapist does not produce an LLM that provides good therapy. It was extremely ineffective and produced an LLM that spoke nonsense most of the time.
Similarly, using retrieval augmented generation (RAG) for an LLM to retrieve instructions from a PDF with instructions on how to give therapy was also extremely ineffective. The LLM often confused therapy examples as the user’s reality resulting in incorrect advice being given.
In the end, I found that the most effective solution was guardrails for self-harm and suicide prevention and effective system prompting using open-source Lang-chain frameworks.
Accomplishments that we're proud of
Proud of having developed a mental health chatbot which prioritises user safety and data privacy (open source LLM prevents user data being sent to a third party). User safety prioritised via multiple checks for suicide risk, however cannot be proven to be fool proof. Hence why this is not a replacement service for psychologists.
What we learned
A lot about langchain, prompting, fine tuning, retrieval augmented generation and when these are appropriate and when they are not.
What's next for My PANDA Hackathon Project
More refinements to the web app and interface.
Log in or sign up for Devpost to join the conversation.