Above demo video contains integration with Spotify. Here is full demo playlist
Dwight vs Google Assistant
For a few years now I’ve been working on nifty and novelty service automations using IFTTT. The idea to use one platform to connect multiple services that I use frequently has always appealed to me and when I found out about AWS Lex’s powerful platform I had to try it out!
To utilise the platform to its zenith I wanted to add as many services as I can and make it as productive as possible. While making the app, I was reminded of Dwight, an assistant in the popular show The Office and given how my use case fits his job description, I christened my app on his name.
What it does
Dwight is a chatbot assistant that helps you connect to multiple services by adding them via OAuth. The services that I have added for now are:
Gmail : Access your mailbox through commands like "Show my unread emails", "Whats last starred mail", "Show mails after 17 july", "Show all emails sent by projjol", Send, forward, reply and other actions on mail
Spotify : Developing this skill was really fun and now I can control my music on Spotify via this new skill. I can search for new music. I can also play specific songs and artists using this skill
Uber: Travelling takes an important part of our day and organising this is efficiently is a major concern for me. Using this skill I can now book an Uber and constantly view the current prices.
- One command and deploy whole stack to AWS. It leverages boto3 client to create/edit/update/delete resources in AWS
- Highly extendable architecture. Its extremely easy to add new services which uses Oauth to give access to user data
- Search mails by keywords, time, name of user who sent it, unread, starred etc
- Play music by artist name, track name and album. All player actions like play, pause etc
- Keep context between channels. Support you are chatting with Dwight on Facebook and you switch to Dwight slack, It will remember your last conversation and continue it
How we built it
I started reading docs about Lex and Lambda. Then I picked best three services I wanted to integrate. I chose Uber, Gmail and spotify as I use them extensively. One by one reading their docs, I integrated it to Lex. All of them supports Oauth2 so I came with simple architecture which supports all and also extendable in future.
Challenges we ran into
One of the biggest challenge was to get Oauth credentials into system. I finally got idea to use APIGateway to set cookie in user browser and redirect to consent page, after that authorization code is passed to lambda by passing initially set cookie(hence mapping same eariler user) and saving all details to dynamodb One major hurdle was to get used to the AWS ecosystem as I was working on it for the first time. A good deal of my time was spent learning the ins and outs of the system and how to use it efficiently
What we learned
I'm not much acquainted AWS, but during this hackathon I learned Lambda, Lex, API gateway and Dynamodb. This was best opportunity to learn more about AWS. I'm also a ruby person so shifting from ruby to python was good learning experience too.
What's next for Dwight
I have integrated limited services and limited APIs in each integration. As we integrated more services and APIs this will be much powerful. I'm planning to maintain this project, any many more servies like Google drive, Youtube, Reddit, Facebook, Twitter, Google Maps etc. For Gmail integration, push notification for incoming mails Letting user interact with bot using speech Developer mode - Dynamic add/modify lex models through boto3 client.