Help I Lost My Keyboard

Spilled coffee on your precious input device but want to play games or finish typing the deadline last minute? Fear no more, we've got you covered.

Comment

Inspiration

Keyboards are very fragile and somewhat limited devices, however crucial to the functioning of every hacker. We decided to try substituting them with a ukulele and explore the unlimited possibilities of communicating through sound.

What it does

We are using the laptop microphone to determine what notes you are playing on the ukulele, in real time! The notes are mapped to keys using tuning information of the ukulele and a specified keyboard layout.

We also have a set of custom made games for you to play using the ukulele as input. Our flagship game is Whack-a-mole, where you can whack moles while playing a beatuiful melody.

How we built it

The project is written fully in Python. We are using pyaudio to read input from your microphone, applying FFT and various post processing techniques using numpy to determine what notes you are playing. We then use pyautogui to simulate keypresses, or use the input directly in one of our custom made ukulele games written with Python Arcade.

Challenges we ran into

The note detection is sometimes unreliable with loud background noise, and overtones are problematic when trying to detect multiple notes at once.

Accomplishments that we're proud of

  • Whacked all the moles
  • Played Twinkle Twinkle Little Start

What we learned

That whacking moles with ukulele is fun

Built With

Share this project:

Updates