Voice Calculator with AssemblyAI

Code environment with emulated android device
Audio Engine
Python Script
Java program to convert string to number or character
Circle?
.xml Framework

Inspiration

We wanted to utilize AssemblyAI in our project. At the beginning we brainstormed some ideas and thought that making an AI powered calculator would be a fun and challenging project. We have been using Java for quite some time and this project was the inspiration we needed to excel.

What it does

Our calculator takes a voice input, computes it, and outputs a response in the form of a double. With the press of a button, our code records to an audio file. The audio file is then sent through the web to AssemblyAI where they receive and transcribe the file. They then return a Json file containing the transcript of your uploaded audio. Using Java we wrote code to interpret the transcript and convert it to numbers and logical expressions. Once it is all converted we computed it by using an api called mXParser. Our final result is outputed on the graphical interface of the calculator app. Also, aside from the voice functionality, it is a fully working calculator.

How we built it

We built it with Java, Android Studio, Eclipse, Python. We split our group and had half working on the back-end coding and the other half working on the calculator app. Once both halves were completed, we spent all night implementing the two together.

Challenges we ran into

Considering we both spent 8ish hours on debugging, many. The largest challenge was getting the python script to run properly in the Java environment. Also, we had a lot of corrections to make within our code to make it run as intended. Our project in total took us about 24 working hours to complete.

Accomplishments that we're proud of

This is our first time submitting a project to a Hacking event. It was a fun experience and we would love to participate again in the future. We are proud of our calculator and hope to improve upon its current functionality. I, Darren, am proud that I was able to write all the back-end coding to the app.

What we learned

This project taught us a large array of things. For starters, how to make a Java-based app in Android Studio, how to create a graphical interface, how to create Python web scripts, how to record and process audio files in java, and much more.

What's next for Voice Calculator with AssemblyAI

If we continue the development of our app, it will primarily be bug fixes. However, we would like to add more operators to it as we currently only support : *, -, +, /.

Hack K-State was an amazing learning experience for both of us and we plan on coming back next year. See you all in 2022!