Each year, an average of $300 million of costs related to unintended retained foreign objects (URFOs) is generated. This is the direct result of 1,500 yearly cases, directly linked to URFOs. Studies done by The Joint Commission provide data suggesting the root cause of URFOs is failure to communicate patient information and failure to perform accurate instrument counts. Data also suggests that at least 80% of cases diagnosed with URFO could be prevented. Preventing this starts with using the proper documentation. Currently, patient information - within the operating room - is written on an old-fashioned whiteboard which includes: patient name, date of birth, surgeon's name, procedure, start time, and the date. This is why we've created TrackOR.

What it does

Hands-free documentation, in conjunction with natural language processing, allows us to display not only standard general patient information, but also additional vital data such as: allergies, heart rate, oxygen levels, blood pressure, and temperature. Additionally, TrackOR is able to track instruments actively being used in the operating room. It is done on a seperate side of the display. When the surgeon is ready to begin the procedure, they simply tell TrackOR to “start the procedure”. Following this command, a start time is displayed. At this point, the surgeon proceeds to list the utensils he plans on using and TrackOR systematically places icons on the display. Icons that are highlighted in red, are currently in use - active. Icons not in use, but present on the operating table, appear in grey. The surgeon is constantly able to add or remove instruments to and from the display. Once the surgeon notifies TrackOR that the patient is being closed up, TrackOR is able to alert the surgeon if an instrument has not yet been removed. Ultimately, we are able to reduce cases of URFOs by performing precise instrument counts.

How we built it

We used Dialogflow to handle the user interactions with TrackOR. We trained our software to record all of the important patient data and surgery information without interrupting the normal routine of a surgeon. We have also programmed TrackOR to keep track of what tools are in use and warn the doctor if they say they are closing a patient backup with tools still in use. Our backend and Dialogflow webhook are Node.js apps and it’s connected through Mongoose to our MongoDB. Our frontend was built using VueJS and lit css.

Challenges we ran into

We encountered problems when trying to record multiple of the same surgical instrument through voice. This does not allow a doctor to say he is using five scalpels. A challenge we had was connecting everything together and debugging Google’s Webhook as it’s so new and there’s no console on what’s being returned. We also ran into trouble with our live vitals charts because they were being overwritten by VueJS is certain ways. We fixed that by using asynchronous API calls for patient data.

Accomplishments that we're proud of

We are proud of our software because it can save lives, as well as prevent thousands of people having unintentional objects sewn into them. Additionally, it is able to help out hospitals with many direct (but also indirect) costs related to URFO cases.

What we learned

To solve the problem of recording multiple tools simultaneously, we used composite entities. Normally, tools are recorded as single entities and as a list within the certain entity. Composite entities is a new feature of Dialogflow, allowing you to have entities, each containing multiple sub-entities within it. This is a powerful tool that will allow us to interpret the natural language of doctors more precisely.

What's next for TrackOR

We plan on watching and recording a real surgery, in order to improve our natural language processing, as well as create icons for other surgical tools. We also believe that speaking with a surgeon and fine-tuning to the surgeon’s preferences will create a better product. We will also add more patient data such as glucose and blood sugar readings.

Share this project: