As an effort to keep people’s awareness of how serious COVID-19 has became, we want to create an app that offers users quick access to the up-to-date number of cases (infected and death) caused by COVID. COVID Tracking API (Tracking API) is established and well maintained. Regardless of Tracking API’s great resource on COVID, there is not yet a quick way to obtain data from the API because of its vague parameters and the lack of an interface. Therefore, we are building a COVID Tracker Bot using Wit.ai to provide a user-friendly way for everyone to interact with the API.
What it does
COVID Tracker Bot provides a user-friendly interface and meaningful interaction for ExpDev07's tracking API, which provides up-to-date data about the world's COVID cases from three sources: John Hopkins University (JHU), CSBS, and New York Times. The chatbot app allows Facebook (FB) Messenger users to get instructions by typing
Get Started, etc., and query COVID-19 information by typing in the country names and/or date. Once users send out an input, the bot would detect keywords (getting started, country name, date-time), get the right information based on the keywords, and respond to users correspondingly.
How we built it
The bot is written in Python using FastAPI. Integrated into FB Messenger, COVID Tracker Bot uses Wit.ai for user input analysis. We use Wit
intents model to determine user's action, user needs help versus user needs information, and built-in entities,
wit/datetime, to obtain parameters needed for the Tracking API calls. The app is trained to mainly recognize countries and time.
In terms of architecture, the bot interacts with three external services: Wit.ai, FB Messenger, and Tracking API. First, after receiving the FB message through POST call, the chatbot feeds the raw text content to Wit.ai using Wit client. Secondly, the trained Wit model extracts the message’s intent and entities. Thirdly, the data received from Wit is used to obtain COVID data from the tracking API. Finally, the chatbot processes and returns the data by replying to the end-user.
Challenges we ran into
We faced lots of challenges trying to understand and troubleshoot the interaction between the chatbot and FB Messenger. In the beginning of the Facebook app setup, the payload received didn’t have a text field because of security reasons of an in-development app. We figured that adding test users helped solve this problem.
At one point during testing, the replies didn’t get back to end-users even though the bot receives multiple HTTP calls fom FB. After some researching and helps from the Facebook Online Hackathon community, we realized that FB messenger mechanism only allows
200 responses, categorizing any other responses as
500, and leave the unprocessed messages in the queue. Thus, we implemented better logging and exception handlers to ensure that the chatbot always returns a
Wit.ai’s built-in entities helped us extract crucial data to get COVID cases from the Tracking API, which requires two-letter country codes. However, challenges arose when we tried to build the lookup's mapping system from country names to country codes. Wit doesn't refer to any documentation about resolved
wit/location entity's values, e.g. "USA" is resolved into "United States of America" by Wit. Our solution was to run a dataset consisting of country names and codes in Wit to obtain the exact string value, and map them to the corresponding country codes.
Accomplishments that we're proud of
We learned about this Hackathon when it was just one week away from the submission date. We are proud that we promptly came up with the COVID Tracker Bot idea and were able to implement it, though none of our team members ever built a chatbot before. We consider our ability to learn and apply a great amount of new knowledge while doing this project a big accomplishment.
What we learned
We gained in-depth knowledge and hands-on experience in NLP, processing data, test-driven-development framework, and how to build a chatbot. Always assuming a very specific, well-designed chatbot framework is a must to make one, we have learned that a secured implementation can be as simple as having an API/microservices with
POST endpoints, especially with the help of Wit.ai and Facebook Messenger authorization protocol.
What's next for COVID Tracker Bot
For now, the bot only supports John Hopkins University’s data and process input with country names and/or a specific date. We’d love to add the other two sources (CSBS and New York Times) in the future, which provides information on US states and regions, as well as include more functionality to process interval dates/multiple time values. At the moment, our processing time is a bit slow; it is taken up by the bot’s calls to Tracking API. The delay might be even longer (up to 30 seconds) if the data isn’t cached in the Tracking API. Therefore, instead of implementing the chatbot as an app separately, it can be an extended feature in the currently open sourced tracking API (a.k.a. a bunch of additional modules and two more endpoints). The transition is feasible since both the bot and the API implements FastAPI. Lastly, our main goal for next steps would be scaling and expanding the bot, specifically using Facebook Messenger Quick Reply to train Wit.ai and integrating the bot into other social media platforms like Twitter.