• 

CNN-Bi-LSTM for SARS-CoV-2 Classification

Aidan Harbison (aharbiso), Corinn Tiffany (ctiffany), Rachel Chae (rchae2), Zyn Yee Ang (zang)

Final Write-Up

Code

Check-In 3

Poster

Initial DevPost submission:

Introduction

What problem are you trying to solve and why?

We are trying to solve the problem of accurately identifying SARS-CoV-2 virus from other Coronaviruses using deep learning methods to complement the existing molecular testing techniques, which have been found to be inaccurate and unreliable. Accurate identification is crucial in controlling the transmission rates and limiting potential consequences caused by the virus.

If you are implementing an existing paper, describe the paper’s objectives and why you chose this paper.

The paper "Deep Learning for SARS COV-2 Genome Sequences" by Albert Whata and Charles Chimedza aims to explore how deep learning methods can be used to accurately identify SARS-CoV-2 from other Coronaviruses and to detect regulatory or transcription motifs in the virus. The authors chose this topic because the detection of SARS-CoV-2 can be challenging due to genetic similarities among viruses in the Coronaviridae family, and the accuracy of current detection methods is questionable. By using deep learning methods, the authors hope to complement the existing molecular testing techniques to improve the detection rates of SARS-CoV-2 and gain valuable knowledge about cell processes that could help better understand mutations that may affect the regulatory mechanism of gene expression.

What kind of problem is this? Classification? Regression? Structured prediction? Reinforcement Learning? Unsupervised Learning? Etc.

The problem of identifying SARS-CoV-2 virus from other Coronaviruses is a classification problem, where we are trying to classify a given sequence as either belonging to SARS-CoV-2 or not based on its characteristics and features. The problem of identifying regulatory motifs in DNA sequences is also a classification problem, where we are trying to classify a given sequence as either containing regulatory motifs for the SARS-CoV-2 virus or not based on its nucleotide composition.

Related Work: Are you aware of any, or is there any prior work that you drew on to do your project?

Other than the study we are reproducing, no.

Please read and briefly summarize (no more than one paragraph) at least one paper/article/blog relevant to your topic beyond the paper you are re-implementing/novel idea you are researching.

NCNet: Deep Learning Network Models for Predicting Function of Non-coding DNA This paper discusses the application of deep learning to bioinformatics, utilizing various deep neural networks like CNNs and RNNs to predict protein binding and accessibility based on a DNA sequence, noncoding function de novo from a sequence, the accessibility of DNA sequences, and protein subcellular localization. The authors propose several enhancements of the convolutional part in the hybrid framework proposed in the DanQ model. This paper is relevant because the use of a CNN-Bi-LSTM model in the paper that we will be implementing was inspired by this earlier paper’s successes in modeling DNA sequences. For example, this paper mentions that CNNs have outperformed machine learning algorithms that include support vector machines (SVM) or random forests in predicting protein binding based on DNA sequences.

Data: What data are you using (if any)? How big is it? Will you need to do significant preprocessing?

We will be using the NCBI genes database of coronavirus gene sequences as the basis for the data. The data will be accessed using the Entrez library implemented in BioPython, which will allow us to easily access the data in the form of GenBank or FASTA files in our code. Preprocessing will include converting the raw data formats accessed by Entrez into processable data (probably using SeqIO in BioPython). Here is a link to a Colab notebook we authored about how to complete this. Afterward, we will remove any duplicate, non-RNA sequences, and then classify each sequence binarily, based on whether they are SARS COV-2 sequences or not. Sequences will be preprocessed into k-mer “words” of size n=3, creating a 2D feature matrix that maintains word order that is used as the input to the CNN.

Methodology

What is the architecture of your model? How are you training the model?

We will be following the model architecture described in the paper. We will one-hot encode the genomic data, then apply 3 convolutional layers, max pooling, and bi-directional LSTM using concatenation to combine the forward and backward layer, followed by a fully connected classification layer. We will use dropout in the LSTM and after the convolutional and max-pooling layers. We will use binary cross-entropy loss and the Adam optimiser with a learning rate of 0.001 for 1000 epochs.

If you are implementing an existing paper, detail what you think will be the hardest part about implementing the model here.

The most difficult part about implementing the model will be hyperparameter tuning if those described in the paper do not perform well. We may have to adjust learning rate, hidden layer sizes, batch size, or drop-out rate.

Metrics: What constitutes “success?”

What experiments do you plan to run?

We will gather a test dataset from the NCBI genes database to evaluate our accuracy metrics. If the same hyperparameters as that in our model do not perform as well as theirs, we will run experiments with different learning rates and other hyperparameters.

For most of our assignments, we have looked at the accuracy of the model. Does the notion of “accuracy” apply for your project, or is some other metric more appropriate?

Accuracy can work as the evaluation metric, but the study suggests that accuracy would be biased for an unbalanced dataset. They used Sensitivity, Specificity, Precision, Accuracy, Mathew’s Correlation Coefficient, Cohen’s Kappa, and No Information Rate as their metrics to evaluate the model.

If you are implementing an existing project, detail what the authors of that paper were hoping to find and how they quantified the results of their model.

The authors were hoping to build a model that can classify if a given genome sequence is COVID and also detect if a sequence has candidate regulatory motifs or binding sites. Their CNN-Bi-LSTM performed very well by the following metrics:

test	score	description of metric
Sensitivity	99.97	percent true positives
Specificity	99.97	percent true negatives
Precision	99.92	ratio true positives to all positives
Accuracy	99.95	binary accuracy
AUC ROC	100.00	Area Under Curve Receiver Operating Characteristic
Matthew’s Correlation Coefficient	0.9978	model evaluation tool - measures the difference between the predicted values and actual values
Cohen’s Kappa	0.9978	metric of how well the actual classes and the classes predicted agree

What are your base, target, and stretch goals?

base: Implement the CNN-Bi-LSTM model to classify the coronavirus samples as SARS CoV-2 or another coronavirus using accuracy with a balanced dataset as the evaluation metric.

target: Implement the CNN-Bi-LSTM model to classify the coronavirus samples as SARS CoV-2 or another coronavirus, then use the study’s full set of evaluation metrics: Sensitivity, Specificity, Precision, Accuracy, Mathew’s Correlation Coefficient, Cohen’s Kappa, and No Information Rate.

stretch: Use saliency mapping to determine whether a sequence contains candidate SARS CoV-2 regulatory motifs, and if so where they occur in the samples

Ethics

Who are the major “stakeholders” in this problem, and what are the consequences of mistakes made by your algorithm?

It is essential that different types of variants are identified as it is essential to alerting scientists of new types of symptoms that directly affect the human population. It is integral to recognize variants as widespread infection of those like delta and omicron has spiked the deaths in children and strengthened symptoms. Mistakes in algorithms by a lack of thorough analysis would be putting human lives at risk. Stakeholders are not only limited to groups of experts and researchers who characterize emerging variants and monitor impact of vaccines on human health, but it is also essential to also notify local and state public health officials who make decisions on safety regulations for the public. Such stakeholders are essential to change human behavior to deescalate the spread of the variant. Additionally, rapid recognition of variants also can alter the speed and development of vaccines, therapeutics, and diagnostics which are led by biotechnology and pharmaceutical companies.

Why is Deep Learning a good approach to this problem?

Deep Learning is essential to identify patterns and produce classification algorithms as models like convolutional neural networks (CNNs) and recurrent neural networks (RNNs) can be applied to genomic data to identify patterns and differences amongst variants. Such models for our problem can help us identify even new COVID-19 variants as they emerge since deep learning models are able to adapt and generate from data through continuous training. Since deep learning algorithms are well suited for processing genomic data, they can use genomic data and sequences to store and remember crucial information to generate predictions and classify variants from the data.

Division of labor:

We will be having weekly meetings to divide up the subtasks to achieve our goals for that week. We will assign point people for different aspects of the project as they arise. We are planning to share the programming and report writing tasks equally.

Built With

• keras
• python