The Big Picture
Reaching the SDGs requires behavior change, from shifting the transportation and consumption habits of people living in cities to the adoption of new irrigation and crop rotation techniques by rural farmers. Accomplishing this level of change will require the rapid development and adoption of new tools, channels, and strategies by the organizations that are working to drive this kind of behavior change.
Yet these organizations (collectively referred to as the social sector) are outcomes-oriented and often slow to adopt new tools, strategies, and techniques to further behavior change. While the fastest growing and most influential segments of society are driving the adoption of products and ideologies, by moving fast and remaining nimble, the social sector is stuck using old techniques and channels, many of which are becoming less and less effective in our mobile+social-first society. Innovation and adoption of new channels and strategies remain a distant dream for many organizations working toward the SDGs simply because they have no way to measure or evaluate the efficacy of the new approaches.
The tech sector has an important role to play in helping the social sector develop and adopt new tools, channels, and strategies. But if we fail to provide means for these new approaches to be measured and evaluated, these efforts will be in vain.
The current methodologies for measuring, evaluating and learning are staggeringly slow and expensive -- often costing hundreds of thousands of dollars and months of research time. By contrast, this project has demonstrated what can be accomplished with a simple research bot coupled with Facebook ads, Facebook Analytics, and a few AWS services.
Our team interviewed an SBCC public health researcher from UCSF during the hackathon who pointed out that social change behavior change campaigns often spend up to 80% of the project budget, and many months on research, data collection, and evaluation. In other words, organizations regularly spend more money trying to measure change than on change itself.
There is a better way.
The pilot project is highly promising. WhatsApp and Messenger based chat-powered research provides key benefits, including the fact that it leverages Facebook data, easing the collection process, particularly for critical demographic information.
This is measurement, evaluation & learning via chat, vast, affordable surveys through Facebook.
What it Does
This is measurement, evaluation & learning via chat, fast, affordable surveys through Facebook.
This proves/demonstrates the next steps in Facebook Native MLE (Measurement, Learning, and Evaluation) by:
- Providing a fast and inexpensive way to perform M&E and market research.
- Allowing for measurement that meets the needs of academic researchers in a way that Facebook’s Brand Lift studies do not.
- Creating custom audiences of subsets of people who have completed the survey. These can then be used to generate lookalike audiences for use during the campaign.
- Giving researchers the ability to reach out to people who have participated in the survey/research via direct message for future/follow-on research
The app and delivery method function as follows:
- Use Facebook ads to reach out target audience with Facebook to Messenger ad.
- The user interacts with the bot, answering questions.
- Data collected from Facebook’s API and from users’ responses to questions then getstored in AWS Dynamodb
Because this survey includes information about health, we are not storing any PII. However, Messenger gives us the ability to capture other data that helps us ensure that the person is, in fact, in our target audience. This includes age and location.
- The research can then export the data into Tableau or other data-viz or stats package for analysis.
How we built it
First, find a chatbot library (@aiteq/messenger-bot) which is designed for conversation. We want to serve dynamically surveys, so assume that our logic should come from outside( ex. Database, Cache or API). We focused on developing dynamically conversation. After that, we made storing data into AWS Dynamodb with SQS and lambda.
Challenges we ran into
First, Bot library has some bugs which we didn't expect. So we include all source code of the library and fixed them. And we will send PR to that library :smile:
Second, The human body is a fragile machine and sometimes.
Accomplishments that we're proud of
DynamoDB, lambda and SQS were not familiar to us, but we studied AWS and finally, we did it. We are very proud of this opportunity to get acquainted with the node and chatbot.
We are proud that we were able not only to build the app but also launch a multilingual campaign in Nigeria to begin to glean insights on the perception of malaria and mosquito abatement among people in Abuja and Legos.
We were able to develop on the initial vision while at the same time overcoming some debilitating personal health issues.
What we learned
This is the first time anyone on our team had built a bot so the list is long.
What's next for Facebook Native MLE (Measurement, Learning, and Evaluation)
Our team intends to continue this project forward. The next step is to extend what we have built here to WhatsApp in order to make it better serve developing countries where WhatsApp usage is far greater than Messenger. We have begun to implement tools to make the creation of research bots easier (creator tools) and will continue that work.