ParaPhrase

The App in development (Android studio in the background).
Example of summary
Example of sentence context
Some of the TextParse file, which manages the processed audio
Start of Speech.py, which takes in an audio file and creates the text output

Inspiration

Whilst out on an emergency, paramedics have a number of responsibilities that can make keeping track of key events a challenge. On arrival at the hospital, doctors need to know what medications have been prescribed and what procedures have been performed so they know where they are taking off from. Miscommunication can potentially be life threatening, leading to unnecessary double-doses, allergic reactions and much worse.

We set out to create a solution to this problem that alleviates the burden on paramedics by automatically keeping track of events throughout an emergency and ensuring that doctors receive the key information they need on arrival at the hospital.

What it does

ParaPhrase is a mobile app for paramedics that summarises everything that happens from when they first reach the scene until they reach the hospital, where they can quickly handover a full breakdown of events.

It records everything from medication prescribed to procedures performed, providing both a timeline of events as well as a breakdown by medicine/procedure.

How we built it

We built a native mobile app that handles the recording of audio throughout the emergency. The app then sends the audio to our servers, which then:

Transcribes the audio files to text, including a breakdown of exactly when each word was said to create an accurate timeline;
Process this audio, looking for key terms using our curated database of medication and common paramedical procedures to classify;
Return the breakdown in a detailed but easy-to-read format.

Our app then takes this data back and displays it in a convenient and easily accessible format.

Challenges we ran into

Linking our mobile app, servers and processing algorithms together proved to be much more challenging than expected. However, planning in advance meant our stack managed to fit nicely together in the end.

Accomplishments that we're proud of

We're proud that we were able to bring together a wide range of data sources, build algorithms to process speech in audio file format and set up the mobile app and server infrastructure to bring everything together in such a short amount of time.

What we learned

We learned that careful delegation of tasks is crucial, especially given the many dependencies involved in the different parts of our project. By the end, we were able to delegate effectively, ensuring that everyone had something to do at a given time and that dependencies would not lead to people having to stop development and wait for someone else to finish.