We were originally going to do a handwriting recognition hack to convert handwriting into Wolfram Alpha queries. When we learned that Amazon was providing hardware to hack with (Amazon Echo), we decided that dictating a query made more sense for a student trying to write homework.
What it does
This hack allows your spoken query to be sent to Wolfram Alpha for solving. While it was optimized for math queries (such as "integrate x squared from zero to three"), it can also answer some general questions that Wolfram Alpha is known for (such as "Facebook stock price").
How I built it
The code is hosted on the Amazon Lambda compute service and is written in Python. It relies on Amazon Echo's built-in voice recognition and Wolfram Alpha's natural language recognition. The challenging part was converting everyday speech into a format that Wolfram Alpha is most likely to understand without throwing errors. Due to the fluid nature of language and the white noise that can affect Echo's understanding of queries, this can be a challenge.
Challenges I ran into
The biggest challenge for us was properly parsing human speech to begin with. While Amazon Echo does an excellent job of identifying the words that are spoken, how it passes those words into our function can vary significantly. Making sure we covered as many cases as possible - and ensuring we could convert those cases into Wolfram Alpha queries - was the backbone of our coding.
Accomplishments that I'm proud of
Our group split into two parts - one working on parsing Amazon Echo speech and one working on getting results from Wolfram Alpha. For each of us, the accomplishment we're most proud of was simply getting our side of the code to work.
What I learned
All of us learned the importance of robust code, as this is one of the biggest drawbacks currently affecting voice recognition. The wide range of possible variable combinations necessitated a great deal of testing to ensure that Wolfram Alexa sent the right queries and returned the correct results.
What's next for Wolfram Alexa
If we get a chance to work with Alexa again, we hope to be able to expand the range of queries it can interpret and pass to Wolfram Alpha - for now, it is limited to relatively basic math equations.