SoulTalk

Summary of Video
Sentiment Analysis
Transcript From AssembleAI
Figma Design Topics

Background

Seeing how people around the globe have trouble accessing mental health services, due to multiple barriers -- whether it be financial, social, etc. We found that traditional therapy can be very expensive for most people, additionally the registration process itself is time and energy intensive and after all that paperwork, there is always the possibility that the therapist didn't "click" for the person. Therefore, we realized that therapy is difficult to access for many people as it funnels out the people who need it most.

Therefore we developed SoulTalk, a self-help program that provides an interactive self-help experience through which the user can select the categories they struggle with and proceed. Our AI gives an analysis of your video diary and helps you reflect on your thoughts and feelings.

What it does

When the user starts an audio, the voice is recorded and stored as an mp3 file. Then the mp3 file gets uploaded as an input file into the AssemblyAI. Subsequently, the AssemblyAI starts processing the audio recording and gives 3 separate outputs; the transcript of the audio file to be shown to the user, the summary/ key points of the session/ audio file determined by the AI, and the sentiment analysis of the user based on the recording. The sentiment analysis gives a "Positive", "Negative", and "Neutral" verdict and the system's confidence rating for each verdict for each sentence said by the user. Based on that data, we compile an overall feeling of the session and represent the data to the user in a circular statistical graphic.

How we built it

We built our app by using React for the frontend specifically the chakra-ui React library. For the backend we used Flask as our local server and we used AssemblyAI's audio intelligence to provide our service with insightful data that would be used in our attempts to provide a more convenient diary or self care site.

Challenges we ran into

One of the challenges Since the AssemblyAI only accepted prerecorded audio files and didn't support live record and upload functionality, we had to find an automated way to upload the mp3 file that the user recorded onto the AssemblyAI for processing. Dealing with both frontend and backend servers was a challange and we were hindered by CORS policy.

Accomplishments that we're proud of

Getting the back end to successfully interact with the front end
Having a fully designed prototype on Figma
Cool Use of Assemble AI for an honourable purpose

What we learned

We learned that working as a team takes effort and coordination.
We learned to work in short intervals and get tasks completed.
We learned to communicate effectively by asking each other questions when we don't understand.
We learned to step out of our comfort zone and experiment with technologies that we haven't used before or are not familiar with.

What's next for Soul

In the future iterations for Soul, we intend to improve Soul [Our AI 🙂 ] and add video functionality which will allow the AI to increase/ fine-tune its accuracy when determining the sentimental analysis of its user. Additionally, we intend to add an option where based on the user's location and the type of help/ counseling they need, Soul can recommend trusted advisors from its database and give the option of reaching out if the user feels comfortable.