Given the increase in mental health awareness, we wanted to focus on therapy treatment tools in order to enhance the effectiveness of therapy. Therapists rely on hand-written notes and personal memory to progress emotionally with their clients, and there is no assistive digital tool for therapists to keep track of clients’ sentiment throughout a session. Therefore, we want to equip therapists with the ability to better analyze raw data, and track patient progress over time.
- Vanessa Seto, Systems Design Engineering at the University of Waterloo
- Daniel Wang, CS at the University of Toronto
- Quinnan Gill, Computer Engineering at the University of Pittsburgh
- Sanchit Batra, CS at the University of Buffalo
What it does
Inkblot is a digital tool to give therapists a second opinion, by performing sentimental analysis on a patient throughout a therapy session. It keeps track of client progress as they attend more therapy sessions, and gives therapists useful data points that aren't usually captured in typical hand-written notes.
Some key features include the ability to scrub across the entire therapy session, allowing the therapist to read the transcript, and look at specific key words associated with certain emotions. Another key feature is the progress tab, that displays past therapy sessions with easy to interpret sentiment data visualizations, to allow therapists to see the overall ups and downs in a patient's visits.
How we built it
We built the front end using Angular and hosted the web page locally. Given a complex data set, we wanted to present our application in a simple and user-friendly manner. We created a styling and branding template for the application and designed the UI from scratch.
For the back-end we hosted a REST API built using Flask on GCP in order to easily access API's offered by GCP. Most notably, we took advantage of Google Vision API to perform sentiment analysis and used their speech to text API to transcribe a patient's therapy session.
Challenges we ran into
- Integrated a chart library in Angular that met our project’s complex data needs
- Working with raw data
- Audio processing and conversions for session video clips
Accomplishments that we're proud of
- Using GCP in its full effectiveness for our use case, including technologies like Google Cloud Storage, Google Compute VM, Google Cloud Firewall / LoadBalancer, as well as both Vision API and Speech-To-Text
- Implementing the entire front-end from scratch in Angular, with the integration of real-time data
- Great UI Design :)
What's next for Inkblot
- Database integration: Keeping user data, keeping historical data, user profiles (login)
- Twilio Integration
- HIPAA Compliancy
- Investigate blockchain technology with the help of BlockStack
- Testing the product with professional therapists