-
-
In Game
-
Start Menu
-
Victory Screen
Inspiration
I really wanted two explore two things: AssemblyAI, automatic speech recognition with excellent accuracy, and Unity, a widely popular game engine. Both of those things came together in this space-themed idea of intentionally messing with text translation of speech to create a unique puzzle-game!
What it does
Players start the game by clicking "Play", then clicking the "record" button on the top left. Say something like "go left" or "go down", and when you're done, click "Send". This will "send" your navigation instructions to the spaceship pilot.
How I built it
First, I set up a way to obtain audio from the user (C#'s built-in Microphone class for PC/Linux/Mac build, and Unity Asset Store's WebGL-compatible CustomMicrophone class). Then, I export it to a .wav file with SavWav. Then I send this file to the AssemblyAI server with C# and, once the file finishes processing, receive the text transcription of the voice command. I then seek out particular directional words in that input and modify them so that the pilot in the spacecraft receives something slightly different than what was said.
Challenges I ran into
Firstly, I wanted to use AssemblyAI's Python supported language as I was most comfortable with it, and even had it working as a standalone program. Unfortunately, Unity does not support all libraries necessary for the python program to work (pyaudio, SpeechRecognition), so I had to scrap all working python files and switch over to the C# version. I also have never really used C# to the capacity that this project demanded, and ran into a lot of trouble with C#'s asynchronous features. Additionally, it was difficult to get the spaceship to move in a grid-like patter on the input of voice, not keypress, which is Unity's default.
Accomplishments that I'm proud of
I was never sure I could actually have a whole game developed in Unity in 36 hours--let alone while incorporating a completely external API like AssemblyAI, and knowing how to work with neither of those tools beforehand. I'm also proud of how much I was able to accomplish despite being solo. I was initially uneasy about choosing to pursue this project by myself, that perhaps I wouldn't be able to make anything at all, but now I know that's simply not true.
What I learned
I learned every step of the Unity game development process, how AssemblyAI handles its speech to text transcription requests, and how to combine these two interests into something fun and unique.
What's next for Galactic Dialup
Originally, my idea was for this to be a two-player game--where the second player is the pilot of the spaceship, receiving only warped text from player. I would love to continue developing into that after improving level design, UI elements, animations, and optimizing the app's performance. I would love to include more challenging warpings of the speech-to-text, ones that require more than one instruction to circumvent. The great thing about AssemblyAI is that I already possess the tool for it--I can fetch whole sentences as opposed to Unity's default key phrase finder.
Log in or sign up for Devpost to join the conversation.