DetectNow-Bot

Voice and Messenger bot for earlier detecting coughing related to COVID-19 from cough recording at zero costs and at the comfort of any place

Comment 1

Inspiration

DetectBot messenger is a landline and messenger bot which uses Artificial Intelligence to detect COVID-19 from cough sound recordings. Our vision is that every coughing person can get tested for COVID-19 at zero costs directly from home. It can be also accessed from a landline without requiring the internet.

In order to help vulnerable societies from all over the world. Since the internet and smartphone penetration is quite low in rural areas and third world countries, we provide easier access to the testing of COVID-19 simply by the recording of your cough at zero cost from a landline in addition to the messenger. We ask users to record 10 seconds of cough alongside some additional optional demographic and medical questions to make accurate predictions about the diagnosis.

What it does

DetectBot is a chatbot built with wit.ai to guide our users step by step for easier data collection rather than visiting a web form. We provide clear and necessary instructions during the usage of the bot to make sure the safety of the user, his/her surrounding, and the devices being used for collection. We also collect all the necessary consent required from the medical and user data privacy compliance rules (GPDR, HIPAA) to make sure that the data of the user stays private, safe, and anonymized.

Chat-bot flow

DetectBot follows Differential privacy approaches. Differential privacy is a system for publicly sharing information of a dataset by describing the patterns of groups within it while withholding information about individuals. We collect biological data (e.g. age, weight, size, gender) and medical data (e.g. temperature, breathing rate) which we use to run machine learning models to detect bio-signals of the COVID19 cough.

Landline services can be accessible from the following numbers,

  • +44 20 7365 7186 (Europe)
  • +1 844-230-5884 (US)
  • +41 800 110 318 (CH)

Or from the DetectNow page,

DISCLAIMER: The diagnostics function is not live yet, as this will require a medical trial first.

How we built it

We built the chat-bot using wit.ai to enable people to interact with DetectNow services using voice and text. For backend development, we have used nodeJS for logic handling of the bot framework and used Facebook messenger integration with that node JS application for the session, dialogue, and model handling. We've used AWS Polly, along with AWS LEX to provide integration of our bot over the telephone helpline in Europe and the USA. We used AWS lambda function for the integration of wit.ai with the AWS services via API communication. We collected 500 crowd-sourced data points from the #codevscovid19 hackathon on which we trained the AI model for the COVID19 classification. The AI model and backend infrastructure is explained in the below diagram, TechnicalArchitecture

Team

We are a group of machine learning experts, doctors, and entrepreneurs from Switzerland, Egypt, Germany, China, Ukraine, India, Pakistan, Greece, and Spain. We initially found together through a Slack group during the #codevscovid19 hackathon and are working completely remotely and expanded with new team members to make it production-ready. During the Facebook AI hackathon, we also enabled the support for messenger and landline channel on top of the web platform

team

Challenges we ran into

  • We heard about the competition two weeks before it's the end of the submission, and therefore we worked very hard as a team to finish the prototype before the deadline.
  • Getting started for wit.ai was challenging for us as this was all new for our team members and there are very few online resources to get started with wit framework and integrating it with other services like AWS etc.

Accomplishments that we're proud of

  • Working on a common mission with a team of machine learning engineers, doctors, and entrepreneurs.
  • We started developing the AI as part of HackZurich, we managed to collect crowd-sourced data with a web application, on which we further trained our models, and to give further easier access we also enabled voice and messenger bot channel.
  • Having built a functional prototype in just three days with functionalities like bot interface, connecting it with AWS services, and with a landline.
  • Learning wit.ai in a short amount of time, we started working on it for almost two weeks before the deadline and we're quite happy with the progress we've made so far.

What we learned

Our team members had experience with building chatbots using Lex, DialogFlow, Azure bot Framework, and Rasa. But this is our first time using wit.ai and it was challenging and time well spent! Besides, integration of AWS services especially Connect, Polly, Lambda, WIT, Node, it was challenging and rewarding to orchestrate a solution that works using all the components.

What's next for DetectBot

We will add further features in the realm of telemedicine, scheduling appointments, and tracking disease spreading. Enabling:

  • easier & earlier diagnosis of respiratory diseases like Asthma, Bronchitis, Pertussis, etc.
  • a clinical trial to get it approved as a medical device

Built With

Share this project:

Updates