A Project of Voice: The Life Story of Not Using A Keyboard

Inspiration

We wanted to play Minecraft with our voices but we realized the accessibility application not only to the game but the computer itself.

What it does

The program allows the user to give voice commands to simulate mouse and keyboard

How we built it

We built it with python and some libraries for inputs and voice recognition

Challenges we ran into

We ran into multiple challenges including but not limited to timing issues between the inputs and the program start ups, Assembly AI being itself meaning slow and liking to add extra grammatical flare such as periods or the same phrase twice, and not being able to tell whether to use direct input or auto gui.

Accomplishments that we're proud of

We are proud of our most in-depth application applied, Google Chrome. We put effort into making things such as efficient short cuts.

What we learned

We learned how to simulate mouse and keyboard and access other applications outside the local directory.

What's next for A Project of Voice: The Life Story of Not Using A Keyboard

Somethings we want to add in the future include simultaneous button presses, combo presses i.e. ctrl+s, optimized voice recognition time, and being able to listen once audio activity comes from the mic.

Built With

• assemblyai
• numpy
• os
• pyautogui
• pydirectinput
• python
• scipy
• sounddevice