Timeseries Classification with HA-TCN for Stress Levels

Julia Windham, Jacob Zeldin, and Juan Muneton

poster Code: https://github.com/jwindha1/dl-final/tree/main/model Final Paper: https://github.com/jwindha1/dl-final/blob/main/deliverables/CSCI_1470__Final_Write_Up.pdf

Introduction

With the increase of people diagnosed with anxiety and high stress levels, it has become imperative to study the biological factors around this anomaly.Being able to measure stress, therefore, may help to address this problem. Although stress has a psycho-logical origin, it affects several physiological processes in the human body: increased muscle tension in the neck, change in concentration of several hormones and a change in heart rate (HR) and heart rate variability(HRV).This research project aims at studying heart rate levels by identifying stages of stress across different time intervals in patients wearing heart rate sensor monitors. In doing so, our goal is to re-implement previous scientific work conducted by Lin et al., 2019 on the paper Medical Time Series Classification with Hierarchical Attention-Based Temporal ConvolutionalNetworks: a Case Study of Myotonic Dystrophy Diagnosis. While this invention focuses on the interpretable diagnosis of myotonic dystrophy from analysis of handgrip time series data, we propose that this model can also be used for time-series data for measuring stress levels. Therefore, we present an HA-TCN model capable of classifying levels of stress based on heart rate monitoring at different time-steps, further demonstrating that this model has applicability beyond its current use. In this project, we take on the challenge of developing a temporal convolutional neural network capable of determining stages of increase heart rate levels with an addition of an attention model that further provides the benefit of identifying relevant behavior indifferent time-steps while decreasing noise in the data and adjusting importance of relevant patterns for the binary classification of stress levels. We show that our model is capable of learning at a fast rate while also providing an accuracy score beyond 90%, letting us infer that this model is another milestone for the inclusion of artificial intelligent models in the healthcare field.

Related Work

As of today, there is no evidence of previous scientific work aimed at this specific problem using an HA-TCN model. However, Luo et al., 2020, developed an LSTM neural network to establish a heart rate prediction model based on multiple influencing factors. The mean square errors of the experimental results showed that an LSTM model with Adam optimization was capable of having low mean square error values compared to other classical models. Attention mechanisms in deep learning were were initially developed for recurrent neural networks (RNN) on end-to-end machine translation applications. As of today, there has been a growing application of attention models in temporal sequence analysis (see Lei Lin, Beilei Xu, Wencheng Wu, Trevor Richardson, Edgar A. Bernal, 2019). A group of researchers previously used Attention models on LSTM for extraction of important features in dialogue detection (Shen and H.-y. Lee, 2016). In another project, Zhou et al. demonstrated that attention layers in a bi-directional LSTM played a crucial role in prediction of time series classification, a crucial mile-stone and reason for the development of our current project. The addition of hierarchical attention networks (HAN) has been shown to be useful in choosing relevant encoder hidden states in document classification of words . Additional research has also shown that attention models under a dual-stage RNN for time series prediction successfully identified relevant exogenous time series at each time step and salient encoder hidden states across time steps (Y. Qin, D. Song, H. Chen, W. Cheng, 2017). In terms of the TCN part of our model, previous studies have shown that TCNs perform better in prediction and classification models over other recurrent networks such as RNNs and LSTMs (S. Bai, J. Z. Kolter, and V. Koltun, 2018).

https://iopscience.iop.org/article/10.1088/1757-899X/715/1/012060/pdf

Data

For the modeling of this project, we used the WESAD (Wearable Stress and Affect Detection) Data Set. WESAD is a publicly available dataset for wearable stress and affect detection. This multi-modal dataset features physiological and motion data, recorded from both a wrist- and a chest-worn device, of 15 subjects during a lab study. The following sensor modalities are included: blood volume pulse, electrocardiogram, electrodermal activity, electromyogram, respiration, body temperature, and three-axis acceleration. Moreover, the dataset bridges the gap between previous lab studies on stress and emotions, by containing three different affective states (neutral, stress, amusement). In addition, self-reports of the subjects, which were obtained using several established questionnaires, are contained in the dataset

We split the data into training and testing sets. The training split accounted 70% of the data, while testing only 30\%. The WESAD database, however, did not contain the binary classification stress level labels we needed for the implementation of this project. We then decided to further our research exploration in studying heart rate levels and created a threshold that defined whether someone was under stress. Based on previous scientific work, a heart rate above 100 beats can represent stages of major motion activity, which is also correlated with the raise of stress levels. Figure 1 provides an example of a sample of a patient's heart rate within a specific interval. The classification is performed if the heart rate is beyond 100.

https://archive.ics.uci.edu/ml/datasets/WESAD+%28Wearable+Stress+and+Affect+Detection%29

Methodology

For the modeling of this project, we used the WE-SAD (Wearable Stress and Affect Detection) Data Set.WESAD is a publicly available dataset for wearable stress and affect detection. This multi-modal dataset features physiological and motion data, recorded from both a wrist- and a chest-worn device, of 15 subjects during a lab study. The following sensor modalities are included: blood volume pulse, electrocardiogram, electrodermal activity, electromyogram, respiration, body temperature, and three-axis acceleration. We split the data into training and testing sets. The training split accounted 70% of the data, while testing only 30%. The WESAD database, however, did not contain the binary classification stress level labels we needed for the implementation of this project. After studying how heart rate levels can be used to indicate periods of stress, we created a threshold hyperparameter that defined whether someone was under stress, which we set to 100. With our labeling scheme, we use a Temporal Convolutional Network (TCN) with multiple layers of self-attention between each convolution step. Since the TCN can only see. the history before a timestep, the TCN model uses causal convolutions, convolutions where an output at time ti sonly convolved with elements from time t and earlier in the previous layer. e. To address the causal convolution's limited history, we use dilated convolutions, where the dilation factor increases exponentially with each layer. In our model, we have 1 input layer, 6 convolution hidden layers, 6 attention layers ,and 1 output layer with dilation factors d= 1,2,4. We then use that to predict for the next timestep whether an individual is experiencing stress (1) or not (0).

Ethical Questions

What broader societal issues are relevant to your chosen problem space?

The problem in question has large implications in the health industry. The identification of stress and anxiety as medical conditions is still a burgeoning field, and medical professionals struggle to diagnose patients who exhibit varying symptoms. While many cases can be identified through communication and simple qualitative exams, doctors and scientists often look for more effective approaches to quantitatively diagnose when one feels stress. Trends and variability in stressed patient's heart rate, taken either via BVP or ECG measurements, are increasingly being viewed as an appropriate supplement to more accurately diagnose anxiety. Our project attempts to use recent advances in Deep Learning to aid medical professionals to lower the rate of misdiagnoses. The fact that we can build a Deep Learning model on this time series data to predict levels of stress could bring attention to how the scientific community can implement AI as a supplementary solution to their treatments and diagnoses of patients with conditions that are just now being researched and explored.

Why is Deep Learning a good approach to this problem?

Deep learning is an appropriate methodology for this problem of identifying stress levels via heart rate data for several reasons. First, given that heart rate measurements (BVP) occur frequently, any individual time series of heart rate data over an extended period of time is quite large. The size of the data increases as more patients are monitored. Additionally, while our labeling scheme was custom-designed, there are certain patterns that lead to heart rate anomalies (especially in the case of stress, where heart rate variability increases over time and peaks tend to be higher), which are themselves hard to classify with the naked eye. Deep learning models are able to examine this large dataset and identify the patterns that lead up to periods of stress in the fraction of time that a human would, and with much more accuracy. Additionally, while cardiologists and other medical professionals have set standards to evaluate anomalies on ECG or BVP data, a rule-based approach to analyzing heart rate over time proves to be difficult given the individual variation of heart rate frequency and intensity by individual. DL models, especially recurrent architectures, can leverage this complexity to their advantage, predicting impending heart rate anomalies in patients through the learned patterns that the model internalizes.

$$Division of work

Data Engineering: Julia Model Building: All