This project started at Hack the North when my team designed the original chatbot on the Microsoft Azure platform. Our goal for the bot was to enable those with disabilities to overcome the barrier of social disconnection and stay in tune with the people they care about. I quickly became attached to the idea of creating a medium for our target audience to stay connected, and along with the extremely positive feedback from the Microsoft team at the event, I was inspired to carry the project beyond the hackathon. I hope to make a number of improvements to the bot as time goes on, some of which will be sourced directly from suggestions from the team I stayed in touch with at Microsoft.

What it does

The main goal of the bot is to super-simplify the process of staying connected with friends over social media. With this in mind, we built the original with the capability of checking in on a particular user across all social media platforms that their name could be found on. The bot also takes a time-frame parameter, so that it knows which posts to analyze. For example, a user could tell the bot "What has Tommy been up to in the past week?". Once the bot creates a list of posts from the past week, it uses a caption-bot API to analyze the images and figure out what's going on in each of them. Finally, it will synthesize these analyses into a summary of the target's activity for the given time-frame.

How I built it

Knowing little to nothing about creating a chatbot, nearly everything was new to me in this project. I learned how to create a bot on the Azure platform, how to integrate with LUIS (Language Understanding Intelligent Service), and how to connect the bot to the Facebook Messenger platform for ease of use. With help from Microsoft mentors at the hackathon, I learned how to make LUIS form an understanding of what the user was asking for in natural language, and send it back to the Azure bot to determine what to do next.

Challenges I ran into

As mentioned, I had to learn nearly everything from scratch on this project, save for the experience I have in C#. I was able to work through these challenges with persistence, mentorship, support from my team, and a substantial amount of caffeine. The hardest part of the project was getting the user's accounts authenticated within Facebook Messenger app. This was solved by redirecting the user away from the app to authenticate their account, then bringing them back to the conversation automatically, returning with a token for the bot to use when scraping the various social platforms.

Accomplishments that I'm proud of

I am most proud of everything I learned during the hackathon. There were some major barriers my team and I had to overcome, and I'm proud to say that we gained a lot of valuable knowledge from the things we built. Even at the points we had to make compromises or accept defeat, we learned new things that helped us carry the project forward until the end. I am also proud to have created an impactful hack that can make a real difference in the lives of our target audiences, and am excited to continue to develop the project as time goes on.

Future possibilities for

During the hackathon, we only had time to implement a check for a user's Instagram activity. We used this as a proof of concept, because we didn't have time to include support for other platforms. I also want to focus on making the bot even more accessible than originally planned. For example, using IBM Watson or similar APIs, the bot could be made available in dozens of other languages, including using CV to support sign-language.

Share this project: