Getting tested for Covid19 is crucial to preventing the spread of the disease. In many countries such as Bangladesh, Myanmar, and Pakistan, testing is not as readily available nor is it as accessible as compared to wealthier countries like the USA or Canada. By taking advantage of our knowledge of machine learning and signal processing, we aim to bridge that gap by providing equal and equitable access to testing, using the internet as a means to reach places where getting access to a Covid19 test might be difficult.
What it does
Our web app allows a user to upload an audio file containing the sounds of their coughing (whether it’s a forced cough or one that’s caused by some unidentified respiratory illness) to our site. Upon pressing the submission button, the audio file is processed, padded, and inserted into our trained model, which then predicts the likeliness that the audio file contains coughing caused by Covid19. This information is then displayed for the user to see.
How we built it
We first scoured the internet for any sort of dataset on Covid19 and audio. We quickly found the Project Coswara dataset from the Indian Institute of Science, containing approximately 1300 individuals’ worth of coughing, vowel phonation, and breathing. Then we downloaded this data and processed it using Pandas to access the csv data that contained each individuals’ Covid19 postivity status (whether or not they had Covid) and using Librosa to process the audio. We converted each audio file into a Mel-frequency cepstrum (MFC) which is an approximation of a human’s sensitivity to sound. Then, we normalized all the MFCs to make them all the same length, and we fed them into our neural network, a relatively simple feed-forward network. We trained our model over our data in in Google Colab, saved it, and downloaded it for use in our web application. We then created our web application to host the model and interface with our users.
Challenges we ran into
Our first challenge was that we were unfamiliar with librosa and audio processing. This meant that our data processing step took much longer than we anticipated. Through intuition, a lot of Googling (thank God for StackOverflow), and a lot of elbow grease, we managed to process all of our data.
Our next challenge was actually training our network on our data. All of our computers are relatively slow, so training our model on our own computer was taking forever. Our solution was to upload our serialized input data and expected outputs to Google Drive, mount it to Google Colab, and train from there. Once training was completed, we were able to download our serialized model, and then mess around with our architecture to test different models.
Our last and final challenge was actually creating our web app. None of us had much experience with creating web apps reliant on a backend component, so we ended up learning Flask in order to create our web app.
Accomplishments that We’re proud of
We’re most proud of actually being able to create our model in the first place. Starting off with this project, none of us had any experience working with Pytorch or Librosa. We didn’t even know if our project was feasible, and we encountered many obstacles along the way. Despite all of this, we were able to create a model with 93% accuracy.
What we learned
We learned a lot about neural networks, signal processing, and most importantly – working together in a team.
What's next for CoughDrop
Our next steps would be to continue working on our model. One of our ideas for our model originally was to implement ensemble learning and stacking multiple models under another neural network, then have those sub-models all inspect different sounds. One model might detect Covid19 in phonation of the vowel ‘a’, while another detects Covid19 in breathing, another in coughing, etc. Then the metalearner would take in all the outputs of those sub-models to form a more informed classification of the patients’ likeliness for having Covid19.
Another thing we’d like to work on would be making our project available to people in other languages. According to the Harvard Business Review in 2012, approximately 1.75 billion people “speak English at a useful level”. This means that the large majority of the world will not be able to access our web app solely due to the issue of a language barrier.