Voice Assisted Automation Framework (VAAF)

VAAF

Inspiration

Voice recognition and automation seems to be a match made in heaven, but is not commonly used (yet). I looked at the different options, limitations & challenges, and eventually developed a simple and effective framework to bring these two technologies together.

What it does

The Voice Assisted Automation Framework (VAAF) provides a framework for using voice commands to launch automated processes. Simply provide the command, and the workflow to be executed. The framework works with the excellent Cortana assistant so no additional software is needed.

How I built it

I built the Framework using UiPath Studio Community. I tried many different approaches, but finally settled for a simple effective workflow.

Challenges I ran into

Microsoft does not allow classic desktop apps to access Cortana. Cortana cannot be accessed directly with selectors. Inconsistent behavior from Cortana.

Accomplishments that I'm proud of

It works! It works! It works!

What I learned

There are endless possibilities with automation, simple solutions almost always have better results.

What's next for Voice Automation Framework

The current version only works with Windows 10 and Cortana (which is great since Cortana is also the most accurate at voice recognition), so a more generic framework for other versions of Windows needs to be developed. If there is a more direct way of interacting with Cortana that would be ideal too. Launching multiple robots at the same time would also be a nice feature.