One of my grandfathers was diagnosed with Parkinson's Disease (PD) and I was able to see the impacts of this debilitating disease first hand. I was inspired to improve inefficients in the patient care cycle, and I was especially interested in the diagnostic timeline.
Parkinson's disease is a neurological disorder caused by the death of neurons in the substantia nigra, the portion of the mid brain that controls motor movement including resting tremors, muscle rigidity, and gait difficulties. Currently, there is no single, definitive test to diagnose PD. Instead, PD is diagnosed through a combination of clinical criteria, which include the measurement of the deterioration of two or more motor symptoms over time. However, these clinical tests only have a predictive power of 90% accuracy, and clinicians need to follow patients for up to 5 years before making a definitive diagnosis.
The modeling of patient voice data offers a potential route to streamline the PD diagnosis process. In addition to developing motor impairment symptoms, approximately 90% of PD patients exhibit speech disorders. This diagnostic method provides several advantages, as this alternative solution offers rapid detection of PD that can be done in a remote fashion through a remote solution. Additionally, this test is extremely cheap as patients only need an audio recording of their speech.
Therefore, my aim was to develop a quick diagnostic test through a machine learning model trained on patient speech data.
What it does
My project is a machine learning diagnostic tool for detecting presence of Parkinson's disease. My model achieves accuracy of 88% on a hidden-test set. This accuracy is competitive with the current diagnostic methods in the clinic and other audio classifiers trained on other PD datasets. Additionally, my model is much more generalizable to any patient speech dataset.
My model performed with superior accuracy when using lower quality audio and using a general paragraph of text when compared with other state-of-the-art classifiers. Other classifiers were trained on data collected with multidirectional mics and included patients recording loud enunciated vowels.
In conclusion my classifier is generalizable to all audio quality levels, speech types, and provides competitive results with other state-of-the-art models.
How I built it
When building my model architecture, I aimed to create a generable classifier to employ the ML algorithm in a remote setting. One popular method to achieve this is using Transfer learning, the practice of using the weights of models trained on other datasets to inform another model. I created audio embeddings from my dataset using transfer learning and used this data as input to my model built with Keras. Building the model required a lot of work, preprocessing the data was not a simple ordeal due to multiple number of audio files collected from each patient and the imbalance of the dataset.
After developing the model, I built the web app using flask and allowed the user to input their own audio files.
Challenges I ran into
The major challenges came with developing my model. Research is ongoing in the area, and many improvements to the state-of-the-art model have occurred in the past decade. I had to find a novel way to make my model generalizable to lower quality audio types. Additionally, the huge imbalance in my dataset, caused a problem in training as the model would guess the majority class for all input data. I wrote random undersampling code to deal with this issue. Another challenge was patient siloing, due to each patient having multiple samples. I wrote custom random split code to achieve a 80% train-test split, as well as k-fold cross val adhering to siloing.
What I learned
Through developing this project I learned to first conduct literature review to understand the current state-of-the-art in the field and the method used to create their models. I then had to develop a ML model that was competitive with these models but also had other crucial qualities. I learned methods such as Transfer Learning, and how I could implement the extracted embeddings as input to my model. Additionally, I learned the importance of patient siloing and the erroneous results that can occur without adhering to the standards.
What's next for Parkinson's Disease Detection
My next steps to improve the project is to implement other remote tests in my model for improved accuracy. These tests include monitoring motor symptoms such as a patient's gait with a cell-phone accelerometer, motor speed with a tapping test, and testing cognitive skills through a memory test. I hope that I can integrate a model using these 4 combined tests that can provide a more wholistic view of patient PD state.
These tests can also inform exercises patients can practice to improve their symptoms.