Inspiration

Therapy is all about creating a trusting relationship between the clients and their therapist. Building rapport, or trust, is the main job of a therapist, especially at the beginning. But in the current practices, therapists have to take notes throughout the session to keep track of their clients. This does 2 things:

  • Deviate the therapists from getting fully involved in the sessions.
  • Their clients feel disconnected from their therapists (due to minimal/no eye contact, more focus on note-taking than "connecting" with patients, etc.)

What it does

Enter MediScript.

MediScript is an AI-powered android application that:

  • documents the conversation in therapy sessions
  • supports speaker diarization (multiple speaker labeling)
  • determines the theme of the conversation (eg: negative news, drug usage, health issues, etc.)
  • transparently share session transcriptions with clients or therapists as per their consent

With MediScript, we aim to automate the tedious note-taking procedures in therapy sessions and as a result, make therapy sessions engaging again!

How we built it

We built an Android application, adhering to Material Design UI guidelines, and integrated it with the Chaquopy module to run python scripts directly via the android application. Moreover, the audio recording of each session is stored directly within the app which sends the recorded audio files over to an AWS S3 bucket. We made AssemblyAI API calls via the python scripts and accessed the session recording audio files over the same S3 bucket while calling the API.

Documenting conversations, multi-speaker labeling, and conversation theme detection - all of this was made possible by using the brilliant API by AssemblyAI.

Challenges we ran into

Configuring python scripts with the android application proved to be a big challenge initially. We had to experiment with lots of modules before finding Chaquopy which was a perfect fit for our use-case. AsseblyAPI was quite easy to use but we had to figure out a way to host our .mp3 files over the internet so that the API could access them instantly.

Accomplishments that we're proud of

None of us had developed an Android app before so this was certainly a rewarding experience for all 3 of us. We weren't sure we'd be able to build a functioning prototype in time but we're delighted with the results!

What's next for MediScript

  • Privacy inclusion: we wish to use more privacy-centric methods to share session transcripts with the therapists and their clients
  • Make a more easy-to-use and clean UI
  • Integrate emotion detection capabilities for better session logging.

Built With

Share this project:

Updates