Grabbing a fast meal? Use this Chatbot to do a quick calorie count before you hit the counter!

If you're squeezed for time, you may be grabbing food on the go, but sometimes you can accidentally make some poor food choices. What if you had a chatbot that could calculate the calories ahead of you walking up to the counter or the drive-through menu?

What it does

This is a chatbot that can query the main entrees at more than a dozen popular fast food restaurants, and reply back with the calories contained ahead of you making a stop and seeing the menu. Here are the main intents that it responds to.

intents

How I built it

architecture The architecture uses the AWS Lex service as it's core, with Slack and Facebook Messenger user interfaces that connect to it. The backend is a Lambda function written in NodeJS, and as part of building for the Lambda function, JSON files with nutritional data are read.

Lex

I used the AWS Lex service to build the bot. That required setting up a series of intents that corresponded to different questions or phrases that the user may respond with. Included with the intents are sample utterances that train the AI models to use the different intents.

The main intent is a dialog that the bot can have with a user to calculate the number of calories for a specific food and drink combination at a single fast food restaurant. Here is what the setup looks like within Lex, and highlights the narrative that will be invoked when this intent is triggered.

lex

There are three slots that are used to capture key pieces of data in the narrative. I'm using the standard Amazon slots that are available within the platform to detect the restaurant name, the food, and the drink. Once these three pieces of information are captured in an conversation, the bot responds with how many calories are in the combination.

Lambda

The business logic is written in Node.JS, and uses the AWS Lambda service for hosting. The intent and slot data is passed from Lex to the Lambda function for each request that the user makes, and the Lambda function creates the response that is interpreted by the bot.

The code written for the calculate calories intent is more complex than the other simple queries like hello and thank you. In the calculate calories section, it can be called in two different modes. First is a validation mode, and once complete is the final calculate mode. The data from the slots is passed into the function by Lex, and the code is responsible for doing the matching and lookup, as well as creating the error messages that will be shared back by the bot.

Food Data

Every Fast Food Restaurant has a website, and a page or widget within it that allows calorie calculation. What these sites don't have is an API that I could integrate with the Lambda function, so instead, I converted the data from the pages into a json file. Once in a json format, I included them into the build package for the Lambda function, and then used them in lookup functions to validate and translate the calorie counts. Each restaurant has approximately 50 different menu items, and once this data was ingested, I tested out through the Lex console on AWS the different combinations.

Slack Integration

After getting the bot working within the AWS Console, I integrated it with the Slack platform. This enabled a popular front-end that can be used.

To begin, I needed to setup a group in Slack - for this application it is https://fastfoodcaloriecount.slack.com/. Once this was created, I then added an App which generated credentials for services like Oauth. This would be needed for Lex to be able to communicate with the app, and I also needed endpoints (a URL) that the two technologies can communicate with one another. The documentation from both Slack and AWS was very useful, and I was able to get it up and running in about an hour.

After getting the two to communicate, I was able to use not just on my browser, but also through the slack app on my phone. That meant that I could lookup calorie count information wherever I was, including at the restaurants!

slackApp

Once I spent time testing, I began to prepare my submission for the app directory within Slack. This would enable any Slack team to be able to add to their implementation, so required improving what I had created into a product. The first step in this process is to build a simple website that can be used as a reference for anyone that has questions with the product. Here is the website, and it provides support information as well as instructions on how to use the bot. I hope to have this through the app store review process soon!

Facebook Messenger Integration

In addition to Slack, I also integrated this with the Facebook Messenger platform. The process was similar to Slack, requiring first the setup of a Facebook page, then an app related to the page. If someone likes the Facebook page, they can then start using the messaging service either on the native site or within the Messenger App on a mobile device. When building the app itself, there are credentials that need to be created to bind together Lex with the Facebook platform, as well as URL's that provide the endpoints to interact.

Messenger

The final working product is available through Facebook and has been published in the Facebook App Store.

Challenges I ran into

I spent time researching API's that could provide calorie information for all of these different restaurants, but wasn't able to find a service. There are commercial databases that are quite rich in detail, however they have licensing costs associated with them. That made much of the effort in this project around scraping data from websites into a machine readable format. It also made me really hungry!

Accomplishments that I'm proud of

It's fun building with new technology, but this service is really helpful to provide people information that may influence eating habits. I know since creating this that I've been more aware of the calories in different restaurants, and have used it myself to get better educated on the options available. Given that the model with these services is that it's free, hopefully others will be able to leverage.

What I learned

Building a chat-bot end-to-end was a great experience, not just the core Lex service, but also how you can integrate with Slack on the front end as well as backend services like Lambda. This enables to get "real world" user experience from people that are using your bot.

Once this bot became active, I started to get actual users for it, and started to track if the interactions were successful. While the AI within Lex does a great job interpreting when to use each intent, what can be challenging is how users phrase different information within the slot. That requires expanding the data in the lookups, as well as writing some additional code within the Lambda service. For example, the difference between someone asking for "12 count chicken nuggets" vs. "12 piece chicken nuggets". While the website only maps to one of these based on the official name of the food, the user means the same thing in both cases and it should return the answer in either scenario.

What's next for Fast Food Calorie Counter

User feedback and usage will dictate how this bot evolves. The framework of tools used collections each of the interactions, enabling a great feedback loop to improve. Some of this is straight forward - users trying to collect data on restaurants or food items that aren't covered. That just takes scraping more websites!

In some cases the interactions can be improved, particularly around "hit rate" for the phrases that people use that might not match how the bot is designed. The current dialog is a little chatty, and some may prefer to be able to provide the question all in one string. This will require some new intents in the Lex code, but will reuse the existing slots and logic that has been written in Lambda.

Built With

Share this project:

Updates