Owing to the recent technological advancements, venoarterial (VA) extracorporeal membrane oxygenation (ECMO) has been increasingly used in the setting of emergency (1, 2). However, it is not an easy task to remember how to assemble the components of ECMO equipment, since the procedure is complicated and the indication for the therapy is limited (3). Therefore, it is important to secure the training opportunity to maintain the skill level in order to respond quickly to emergency situations. Actually, majority of the ECMO centers in United States have training program for ECMO (4). Currently, augmented reality (AR) or mixed reality (MR) has been recognized as a useful method for the medical education (5, 6). Previous works suggested that visual cue provided with AR can reduce user’s task response time and mental effort (7, 8). Therefore, we seek to develop either marker-based or non-marker-based AR application to navigate practitioners to quickly assemble or set up ECMO system that may indirectly benefit emergent patient care.
- Stretch R, Sauer CM, Yuh DD, Bonde P. National trends in the utilization of short-term mechanical circulatory support: incidence, outcomes, and cost analysis. J Am Coll Cardiol. 2014;64(14):1407-15.
- Abrams D, Garan AR, Abdelbary A, Bacchetta M, Bartlett RH, Beck J, et al. Position paper for the organization of ECMO programs for cardiac failure in adults. Intensive Care Med. 2018;44(6):717-29.
- Sin SWC, Ng PY, Ngai WCW, Lai PCK, Mok AYT, Chan RWK. Simulation training for crises during venoarterial extracorporeal membrane oxygenation. J Thorac Dis. 2019;11(5):2144-52.
- Weems MF, Friedlich PS, Nelson LP, Rake AJ, Klee L, Stein JE, et al. The Role of Extracorporeal Membrane Oxygenation Simulation Training at Extracorporeal Life Support Organization Centers in the United States. Simul Healthc. 2017;12(4):233-9.
- Eckert M, Volmerg JS, Friedrich CM. Augmented Reality in Medicine: Systematic and Bibliographic Review. JMIR Mhealth Uhealth. 2019;7(4):e10967.
- Hu HZ, Feng XB, Shao ZW, Xie M, Xu S, Wu XH, et al. Application and Prospect of Mixed Reality Technology in Medical Field. Curr Med Sci. 2019;39(1):1-6.
- Volmer B, Baumeister J, Von Itzstein S, Bornkessel-Schlesewsky I, Schlesewsky M, Billinghurst M, et al. A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks. IEEE Trans Vis Comput Graph. 2018;24(11):2846-56.
- Braly AM, Nuernberger B, Kim SY. Augmented Reality Improves Procedural Work on an International Space Station Science Instrument. Hum Factors. 2019;61(6):866-78.
What it does
Using the camera of your smart devices, the application recognize the AR markers attached to the ECMO components (e.g. centrifugal pump, oxygenator, controller and tubes) and shows you what you do next in the air to complete the setup. You can assemble the ECMO system step-by-step according to the AR-based navigation.
How I built it
- Vuforia Engine in Unity was mainly used to make the marker-based AR application. Unity project file containing C# codes are available from here: https://www.dropbox.com/s/tc1mftegq6p20nn/vMed2020.zip?dl=0
- Icons were selected from icooon-mono as AR markers (https://www.dropbox.com/sh/u57nsewie2u5923/AABnbeU1Uhkf09ZxqfTzwuOHa?dl=0).
- Simulation environment was setup with our original scenario and handmade equipments.
Challenges I ran into
- Possible version mismatch between the latest iOS and Vuforia led to the run-time error with our iPhone, so we changed the platform to android.
- We had trouble finding good AR-markers and finally found that symmetric images were not adequate.
Accomplishments that I'm proud of
Since all the members are clinician and no software engineer is involved, we are proud that we could finish making something that we really needed.
What I learned
We learned it's necessary to do prototype test before going further.
What's next for AR navigation system for priming ECMO
In order to improve the App quality, we will...
- Implement full navigation scenario using real ECMO system (not limited to simulation purpose).
- Use smart glasses (e.g. nreal light) instead of smartphones or handheld tablets.
- Try non marker-based AR with image recognition technology (e.g. YOLOv3).
- Add more function using hand tracking and gesture or voice control.
- Obtain data on the effectiveness of our application and do analysis.