Therapy is a powerful and important process to overcome mental trauma. However, there is a limited amount of therapists. Therefore, when someone experience significant and/or sudden trauma, it is imperative they receive help soon, so waiting one week for the next therapy appointment could be disastrous. Therefore, we build Constructive Care to offer on-demand therapy support to support mental health.

What it does

Constructive Care is a Virtual Reality project that deploys a Large Language Model that we personally fined tune to improve the client's privacy by ensuring that other Large Language Model do not use the client's therapy conversation to train their model. This project offer quick access to a personalized and insightful virtual reality chat bot to provide the assistance needed for trauma survivor.

How we built it

We built ConstructiveCare with Unity VR a large language model using Google Palm. For APIs, we used google's text-to-speech and speech-to-text.

Challenges we ran into

The biggest issue we ran into was Unity, particularly using Unity to interact with API to get and push data. However, after continuous debugging, we were able to successfully get Unity to interact with the API. However, another issue was dealing with sound files in Unity. Creating the C# script for this process proved to be tedious. In the end, we were able to handle the audio file and pass it to and from the API. In terms of our Large Language Model, we ran into fine tuning and prompt engineering issue.

Accomplishments that we're proud of

We are proud of spending 24 hour hackathon learning, debugging, and making memories. This was a difficult challenge, but we learned a lot and created a lot of fun memories among all the stress.

What we learned

We learned more about Unity VR in terms of using Unity to interact with APIs, fine tuning large language model, and handling and passing sound file across Unity and VR.

What's next for ConstructiveCare

In the future, we want do add personalized training about mental-health like meditation-practice and other powerful mental-wellness techniques. In terms of technology, we aim to add better verification that employees can scan their badge and log into their therapy individual session plan. Beyond this, we want to add better awareness and support by implementing an emergency email system that sends notification to alert manager and the cilent's significant other about their mental health score if its low enough.

Built With

  • fastapi
  • google-speech-to-text
  • unity
Share this project: