Title: Insight Patient Data

Team: Insight

HTC Vive Sponsor Award Recipient
Finalist for Overall Best VR Experience

About The Team

  • Brian Cohn, Lead Scientist
  • Ali Marjaninejad, Data Scientist
  • Dilan Shah, Programmer/Project Manager
  • Martin Shapiro, Health Strategist
  • Serhan Ulkumen, 3D Designer/Programmer


Parkinson’s disease is a slowly progressing neurodegenerative disorder with symptoms including tremor, limb rigidity, slowness of movements, and problems with balance. Advancement of the disease can severely impact the quality of life from physical disability to depression. We developed Insight, a VR patient-centric platform for Parkinson’s disease patients and their family. Insight empowers Parkinson’s patients to live a life of their choosing rather than that of their disease.

What it does

Insight augments Parkinson’s patient’s quality of life by integrating into their existing clinical care our VR assessment, management, and health education. Patients typically see a physician to monitor disease progression and adjust medication and rehabilitation therapy at set interval clinic visits based on symptoms. Insight stays with the patient throughout their life, continuously assessing the user between visits and aiding their providers at the clinic.

Our platform draws upon current care via third-party health information such as medical records and movement data collected in VR to create an assessment of the patient’s health status, provide personalized rehabilitation exercises, and guide the physician team in data-driven decision making. Before the patient begins rehabilitation exercises, he/she touches the wind chime in the house which then transfers symptoms over to the virtual world. For the remainder of the experience, the wind chime movement and sound will signify the symptoms of the patient while the patient’s movements will now be tremor free. The patient will then be guided through evidence-based personalized rehabilitation exercises that while improving physical function will also collect data for disease progression assessment. At the end of the assessment, the patient receives an overview of their current health status including medications, Insight’s health score derived from symptom measurements, and an option to contact a physician through telemedicine. This physician will have a report generated by Insight that includes the collected symptom information.

Insight provides a platform for voluntary data collection and we have integrated neural network integration to allow for artificial intelligence deep learning of prognostic and treatment recommendations going forward.

How we built it

The Insight Patient Data platform was built using a combination of Unity game engine and data analysis tools Matlab and Python.

Low Pass Filter for Hand Tremor

To live with a tremor is to live with constant motion, disruption, and often frustration. We designed a tunable Filter that lets a patient have visible feedback of their hand operating completely smoothly, almost as if they have no visible symptoms of Parkinson's. While they may be shaking vigorously, our filter "cures" hand movement in VR. We took a moving average of the last N controller positions and calculated the average position and Quaternion. The SmoothedHandScript that we attached to the user's hand model is responsible for collecting this data and only showing the user the smoothed output, and we expose the window size to the GUI so the VR proctor can dynamically change the filter settings, so the movement is smooth & responsive.


The inspiration for the environment was heavily influenced by one mentor, Hannah Luxenborg, who explained that rather than an art direction similar to a clinic, we should aim for a soothing ambiance. A majority of the models were modeled and created in Maya by Serhan Ulkumen. Using Unity's terrain system, we sculpted a landscape using a heightmap and trees.

Data Analysis and Reporting

Data is useless without Insight. We designed a frequency analysis of tremor that compares tremor in different dimensions (X = triceps/biceps tremor, Y = shoulder tremor, and Z = wrist abduction/adduction), so a clinician can see the specifics of how the disease has progressed or responded to different medications. A script called RawTrialDataSaver is responsible for taking the experimental trial data and saving it to a local CSV file. The appearance of that file is observed by the Python script that uses MatPlotLib to produce a PDF document for the doctor, and a PNG file with a visualization for the patient. For patients who want to use the most optimal medication, the Insight system provides incredible value in comparing treatment outcomes, tracking disease progression, and contextualizing the score of the patient against other patients. We include an in-game visualization of one of the metrics that the patient can observe after the assessment is complete, with a score that ranks them against different levels of normalized tremor behavior

Hardware Used

  • HTC Vive


Asset Store

Tools Used

  • For analysis we used: NumPy and Pandas
  • For visualization: MatPlotLib

Textures for Models

Images, Video, and Media

  • Generated by our team

Thank you for visiting our repo. Please reach out to us if you are interested in learning more!

Built With

Share this project: