Digital Healthcare Explained

AR Experience for Patient Education. Master students create visualizations of their professional know-how.

Comment

Inspiration

Health topics affect everyone; yet many rely on mixed quality Google search (study by Bertelsmann: 58% search the internet before visiting a doctor). With the AR capabilities of smartphones, we let people learn about health with new didactic methods - combining 3D graphics, sound, voice and content interaction. This opens up a new world of widespread and engaging education!

What it does

Patient education works great as an immersive experience directly on the user's smartphones. With Augmented Reality, users can explore 3D content like spines or MR images by walking around them. With the personal explanation of an Amazon Sumerian host, the app connects to many different senses. This greatly improves the learning experience.

For Digital Healthcare Explained, we created an ever-expanding catalog of topics. Through an intuitive interface, users explore topics they care about. After a brief introduction, you select a topic based on a 3D symbol that invites to learn more about it.

The host then explains an animated 3D visualization of that topic - this includes MR images of the brain, a CT of the spine or the heart. Real parts of our body, just waiting to be explored! Walk closer to the objects, move around them and see them in maximum detail!

How we built it

At the university, I'm teaching visualization of health-topics. The "Digital Healthcare" master's program is open to health professionals of various fields: radiological technology, physiotherapy, dieticians, emergency medical response and many more. Part of the education focuses on innovative technology.

Amazon Sumerian perfectly fit the course: it enables health experts to quickly learn about the complex topic of 3D visualization. Additionally, using Sumerian provided a great into in using Amazon cloud services. How to use text-to-speech, combining services and security are all vital aspects of creating these apps!

"Digital Healthcare Explained" is an ongoing project. It contains an ever expanding catalog of topics, which students will continually build & improve as part of the course.

Challenges we ran into

With many students participating, it was a challenge combining all the individual outputs into a single project. The overall aim was to make user interaction a seamless experience. We had to work hard to ensure that all the messages sent by the host and the great variety of animated 3D models fit together!

Accomplishments that I'm/we're proud of

I've decided to create the lecture materials as public blog post. Sumerian is a great tool, and the whole creator community should benefit from the tutorials and guides we used as part of the lecture. The articles have so far already been read more than 600 times! View the Amazon Sumerian blog post tutorial series

Additionally, we decided to publish the app to the Google Play Store. It's one of few ARCore applications already available to users. It's an important milestone to showcase how the future of patient education could look like! Download the AR app from Google Play

What we learned

Additionally, it's amazing to see the results of individual healthcare topics created by the students. With no prior experience of 3D authoring, all participants were incredibly motivated and built awesome results!

What's next for Digital Healthcare Explained

The submitted version already includes several healthcare explanations. However, the lecture is still on-going and more students will submit their projects soon. The best will be integrated to continually improve the app. And it's very likely that we'll repeat the successful experiment to create healthcare AR content with Amazon Sumerian in the next term!

More information

Team: Andreas Jakl, Elise Mandl, Bernhard Ruhrhofer, Gerald Wagner See the app in action on YouTube Download Digital Healthcare Explained for ARCore capable Android phones from the Google Play Store

Built With

Share this project:

Updates