Multimodal Seizure Detection

Repository Link

Writeup Link

Introduction

In hospitals, timely and accurate monitoring of brain function is essential for diagnosing and managing neurological conditions in patients. Electroencephalography (EEG) is a crucial tool for this purpose, providing insights into the brain's electrical activity. However, the complexity and volume of EEG data necessitate specialized expertise for analysis, a resource often scarce in real-time hospital environments.

To address this challenge, we are developing a multi-modal deep learning model trained on EEG signals from critically ill patients. Our goal is to automate EEG interpretation, enhancing diagnostic speed and accuracy while reducing the burden on clinical staff. This innovation aims to support quicker, more informed treatment decisions, potentially improving patient outcomes in intensive care units by leveraging artificial intelligence for real-time neurophysiological monitoring and analysis.

This is a classification task.

Related Work

Multimodal detection of epilepsy with deep neural networks by Ilias et al. proposes a method of combining the several modalities of the network. We will require a modified version of this idea as we are combining 3 things instead of 2. This paper proposes a multimodal architecture for detection of epilepsy with deep learning. It uses SFFTs to preprocess the signals which are fed through CNNs. In addition, they construct an image from the signal and run it over several pretrained computer vision models. Finally, they flatten and combine the two results with a gated multimodal unit to properly ensure the relative importance of each prediction is reflected in the final output. Their model performs comparable to SOTA models.

Data

Our data comes from a Kaggle dataset. We will need to do significant amounts of preprocessing, such as feature extraction from our signals, FFTs, and converting signals into images for our CNN.

Methodology

In the field of deep learning for EEG signal analysis, Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) are the two most popular and effective methods. RNNs, with their ability to process sequential data, are well-suited for capturing the temporal nature in EEG signals. In addition, EEG data can be converted into a visual spectrogram which has spatial data best suitable for CNNs. Lastly, we plan to extract features from the dataset such as EEG amplitude, wavelength, and other data that doctors use in the real world to make their diagnosis. A multimodal combination of these methods will be more robust and interpretable.

Metrics

In our project, the metric of precision is more accurate than accuracy. This is because we are dealing with a medical task where false positives and false negatives have important implications. For us, success means achieving comparable to or improving SOTA precision for such models (Type I and II errors), we will compare these using a t-test. However, doing so may be complicated so we will break down this into different goals.

Base goal: Achieve comparable precision to SOTA models Target goal: Achieve >= precision to SOTA models or a minor improvement Stretch goal: Significant improvement from SOTA models

In addition, we hope to achieve increased interpretability by using our custom feature extraction and seeing how important they are in generating the final prediction.

Ethics

Why is Deep Learning a good approach to this problem? In general, deep learning models are blackbox and hard to interpret, something not good for the medical field. However, by designing a multimodal architecture for this problem, not only will we make the model more interpretable but also increase model precision, something highly important in the field of medicine.

How are you planning to quantify or measure error or success? What implications does your quantification have? We are quantifying our success through model precision, so that we can minimize our probability of Type-I and Type-II errors. This is highly necessary given our project topic, and would therefore be the right choice for measuring the success. The purpose of our model is identifying and classifying harmful brain activity in the form of epileptic seizures, so misclassification holds significant implications and could be harmful to patients.

Division of Labor

We plan to split work equally amongst all three group members:

Amit - Preprocessing/model architecture, implementation, writeup

Aviral - Model architecture, implementation, writeup

Sean - Preprocessing, Model architecture, fine tuning, writeup

Built With

• python
• tensorflow