Alexa Skill & Android App
While watching TV, I can never seem to find the remote when the phone rings! What if I could say "Alexa, ask Quick Remote to pause the TV" and the TV would just pause while I answered that call?! Wouldn't that be cool?! :)
What it does
Quick Remote helps you control your Roku with voice commands! Play or pause the movie. Rewind or fast forward your favorite shows. Start Netflix or CBS News channel. Navigate Roku's menu by saying go left, right or select!
It can follow multiple commands at once and also control multiple Rokus! You can even ask for it to wait for your next command.
No hub or additional hardware required! You will never touch the Roku remote again!
- Enable Quick Remote skill in your Alexa app
- Link to Quick Remote account using login with Amazon -- your email will only be used to link with the mobile app
- Install 'Quick Remote for Alexa & Roku' mobile app in your phone and login with the same Amazon account
- You will be controlling your Roku in no time!
Alexa Skill: https://www.amazon.com/dp/B06XSCT6XT
Help & FAQs: http://www.appestry.biz/qra
Demo Video: https://youtu.be/SGVWIT2ov9Q
How I built it
I built it using the following tools:
- A Custom Skill in Alexa Skills Kit (ASK) receives the user speech input and passes it as JSON to the web service
- Java Servlets on Google App Engine run the web service that receive the speech JSON and pass it on to FCM
- Firebase Cloud Messaging (FCM) receives the requests from the Java web service and pushes them to mobile apps
- Android mobile app converts the push notifications to remote commands and sends them to Roku
- Login with Amazon API is used in Alexa and Android to link the user from Alexa Skill to the user in the mobile app
- Google Cloud Datastore provides the database to store and link all the user accounts
- Roku External Control Guide is used to send the commands from Android to Roku over WiFi network
All of the above tools work together and stay in sync to turn your voice commands from Alexa to remote control actions in Roku!
Challenges I ran into
Several in fact! I built the Google Home version of this same app using API.AI, which provided a good natural language processing backend. Alexa Skills Kit (ASK), which after all these years still doesn't provide a robust way for free speech recognition or dialogue error handling has a lot of catching up to do!
For example, to recognize "any given" channel name ASK requires LITERAL with a ton of training data, which still doesn't convert "everything" that is said to text! So, recognizing any given channel name is still a challenge.
Also, I couldn't add a UK English version of my Skill because LITERAL is only available in US English. I was told to use custom slot instead of LITERAL, which still doesn't return everything! I'm working on getting all the channels in UK to build a custom slot with enough training data.
Next, if Alexa doesn't understand something it simply matches it to the first Intent listed in the Sample Utterances file and passes null for all its slots, instead of handling the dialogue error inherently! There is a re-prompt feature, but it is only used for silence! Context sensitive error handling quickly became a challenge. At one point, I had to hack my backend logic because Alexa would invoke the wrong intent. I had to pass the slots around from one intent to another for a more accurate contextual error response. Again, the intents that Alexa mixed up had very distinct slots and utterances that did not overlap, so there was no reason for the mix-up to happen!
The Alexa mobile app or alexa.amazon.com website, which is used to link external accounts do not work with pop-up windows! This means I couldn't use any third-party Identity Provider (IDP) that pops up a login dialog to generate my own access token including Login with Amazon (LWA), which became a challenge! The only way to do this is to either manage my own login page/IDs or let Alexa and LWA directly exchange the tokens as described in this very helpful step-by-step blog post. The downside to this approach is that for every new session from Alexa to my backend I have to use its token to hit LWA each time to identify the user to locate their mobile device. This round trip to LWA overhead could be avoided if I managed my own access tokens with Alexa.
All of the above was a breeze in the other voice assistant. Then came the "Skill Certification Process", which was biggest challenge of all!
The testers of Alexa Skill don't fully understand any of Alexa's limitations, which I don't expect them to, but they would simply point me to the submission checklist and not tell me what the fix is for the issue! This quickly became a challenge after a couple of rejections. The most challenging part of the Skill Certification Process (or just about any App Certification Process in Amazon) is that their testers don't fully test the Skill or App before rejecting, instead they would reject it based on the first issue they encounter (even if it is not a blocking issue) and report only that issue, which meant I had to fix each issue separately and resubmit my Skill repeatedly for them to finish testing and publish the Skill.
Overall, developing an Alexa Skill and getting it certified is not for the faint of heart given all of Alexa's restrictions and Amazon's poor developer support!
Whew! I'm glad I got through all the challenges and published the skill. I wouldn't have done it if not for this Skills Challenge of challenges!
Accomplishments that I'm proud of
In 2015 as soon as ASK was released, I tried to publish a Phone book Skill way before they had a LocalBusiness Intent. At that time the testers asked me to list tens of thousands of business, town and city names and zip codes in the US for LITERAL slot! There was no free speech recognition then. I quickly gave up on Alexa (as well as Amazon) and moved on. My Amazon Echo just became a toy for my kids to quiz!
Two years later I'm very proud of the fact I got back into it and actually published an Alexa Skill, albeit most of the same restrictions still apply to Alexa, and I was able to get it done in 2 weeks!! Thank you, Devpost for the motivation! Feels good to check that off my bucket list. :)
What I learned
- Login with Amazon API for both web and Android, which is actually pretty cool!
- Better understanding for Custom Skill in Alexa Skills Kit
- Better understanding for conversational dialogue interface, which I had to manually build for Alexa unlike in API.AI
- To be patient and persevere through the Skill Certification Process
What's next for Quick Remote for Alexa & Roku
I look forward to doing the following, not necessarily in this order...
- Publish the UK English version of the Skill
- Publish the iOS version of the mobile app
- Deep link commands inside a Roku channel, so you could say "Alexa, ask Quick Remote to start CNN on Sling TV" or "Alexa, ask Quick Remote to start Walking Dead on Netflix from where I left off"
- Search for shows using voice in Roku and within channels
- Build a macro feature where users could send a sequence of commands in one shot
If you have any questions or comments please leave them below.