CreScEntS

(C)ont(r)oll(e)r / (S)ensor-based Data-(c)ollection from (Ent)ertainment (S)ystems Means ‘growth’

Inspirations:

We enjoy gaming, and we were interested in what could be done with gaming in general to help the world.

While exploring the available hardware, we were interested in the Xbox adaptive controller as it allowed multiple kinds of control to accommodate any challenges of people playing video games.

More specifically, we realized the wide variety of inexpensive health-monitoring devices that are on the market to help people monitor their health, which provides the basis for health analysis backed up with data.

Problem:

How do we better standardize and generate data about the motor functions/processes of those who are cognitively impaired, or those who are affected by neurodegenerative diseases, to be more accessible for the purposes of diagnosis and research?

Our solution:

• Use an entertaining method of gathering data from patients-- gaming!

• Gather controller-manipulating data from the Xbox adaptive controller as well as physiological data (heart rate, brainwaves) when someone plays a reaction-based game, upload it real-time to a CosmoDB hosted on the Azure Cloud, and ‘visualize’ it on a web app also hosted on the cloud.

• We hope that using a comparison of this data to a dataset of those who are not affected by such impairments as a function of time/onset of the disease will allow more specific mapping and documentation of the disease;

  • One way this data could possibly be used would be to better diagnose such diseases in early stages by analyzing and standardizing the data collected from players in earlier stages of certain diseases.
  • Another possibility would be to correlate the data gathered to regions of the brain, and better map the effects and symptoms of the disease (especially related to emotional and cognitive functions).
  • Of course, the specifications and conclusions of this data would be better determined by neurologists and psychologists, but allowing this data gathered from multiple hospitals and patients to be publicly accessed will allow a much faster, simpler, and extensive accumulation of a dataset for further analysis.
  • Eg. Alzheimer's has a very complex progression of symptoms. Being able to predict the onset of the symptoms would be beneficial to the treatment as well as research on the prevention of such symptoms.

Difficulties faced:

Time management was a significant challenge given the limited 36 hour period and since we are a team of beginners, we needed to develop a project suited to our skills.

• We addressed this by defining our goals and splitting up the work.

Given our project goals, we had to learn to integrate seemingly incompatible devices and collect data from them

• We learned to work with online resources, merge them together, and develop a unique result.

Benefits:

This data collection can be extended in a variety of ways:

• Different games; since there is already such a huge variety of games that are available to play, each type of game can provide a different dataset that would be valuable for different areas of research. As of now we only have a very simple game to demonstrate the collection of data, but with advice from specialists we can easily branch out to more complex (and more fun!) games for patients to play and provide data for.

• Different types of data;

  • The adaptive controller allows many different types of controllers to be added, which in turn can allow different types of physical data to be collected; for example, different parts of the body can be used to press different ‘buttons’ such as foot pedals (which would correlate to foot/leg control), thumbstick-type buttons (which would correlate to overall arm/hand control), auditory sensors (which would correlate to vocal functions), and etc.
  • Different sensors would also allow a wider range of data to be collected; brainwaves, eye tracker/motion sensing, vocal sensing, etc.
  • As mentioned before, different data can be collected based on the type of game being played; a more strategic game would focus on the decision-making functions of a player, while reaction/control based games would focus on the concentration/body control functions of the player.

This allows the entertainment of patients at hospitals as well as mental stimulation which may delay the progression of some neurodegenerative diseases; at the same time, this allows widespread collection of data from multiple different patients so that a more accurate dataset can be collected.

The publicity of the data will allow much easier access to this data, allowing a faster progression of research around these areas.

Once this system is implemented at a certain location, it will not cost much to maintain and expand as the infrastructure is already there.

It also allows a nice convergence of the gaming and medical industries!

Challenges:

Cost/human resources:

• Further development implies that expert opinions are required, especially if this can help with medical diagnoses and documentation.

• Setting systems up at hospitals will require some work, funding, and acceptance by the hospitals

• Data collection

  • Secure and consistent data collection will be a little bit of a challenge because this requires:
  • Consent from the players for data collection
  • Consistency in the ‘account’ used by each player
  • Consistency in the frequency of the player playing the game
  • Removal of false data that is being collected for any reason
    • Fake accounts being made;
    • Playing on more than one account;
  • Data mining and machine learning will help the prediction and categorization of data, which require a large dataset to be both accurate and precise.

Conclusion:

We hope that this project will allow a faster and more accurate progression of medical research regarding the progression and treatment of multiple neurodegenerative diseases, whilst providing an enjoyable and entertaining experience as well as cognitive stimulation in the process.

Built With

• azure
• java
• javascript