We were playing the game "Keep talking and nobody explodes" and wondered if the concept could be applied to medicine.

What it does

OperationVR is a Virtual Reality game in which one player is interacting with patients and observing symptoms, while the other player is giving them information from a general guide book on what might be wrong and what tests to run on the patients in attempt to diagnose and treat the condition.

The conditions range from trivial heartburn to aortic dissection. Players must decipher what the correct treatment is from ambiguous symptoms and methods similar to differential diagnosis all the while communicating with each other and even with the patient themselves using voice recognition.

The goal of OperationVR is to provide an engaging VR experience that teaches aspiring medical students how to communicate under pressure while fielding medical situations.

How we built it

We used aframe to construct the virtual reality game. We created a common data structure to describe surgeries, so that our framework could support multiple conditions. We used dialog flow for natural language processing to simulate physician-patient conversation and use that dialog as a means of aiding diagnosis through voice recognition.

Challenges we ran into

We had issues with being able to pick up virtual objects and use them to interact with other objects.

Accomplishments that we're proud of

We're proud of making the virtual reality game look good.

What we learned

We learned a lot about different diagnoses and the complexity in creating a decision tree to determine diagnoses based on several ambiguous symptoms.

What's next for OperationVR

Adding in more conditions for players to diagnose, increasing the complexity of the game.

Share this project: