Super MIDIo Bros.
Eashwar arrived to the hackathon after the other three members of the team had already settled into their spot (right next to the snack booth, conveniently). He had brought his MIDI keyboard with him since he came directly from a meeting for the Video Game Music Club at the University of Michigan (of which he is a leader). The brainstorming discussion we held started to revolve around the MIDI keyboard and what possible hacks there could be involving it. We settled on the idea of controlling Super Mario Bros. using a MIDI keyboard after Eashwar started playing the Mario theme on the keyboard.
The first thing we had to do was make sure we were able to read in the MIDI input from the keyboard. We experimented around with different libraries and even just directly reading from the USB buffer, but after some exploration we settled on using
pygame.midi for our MIDI input. We also settled on
pyautogui as the library to use to give Python the ability to manipulate the emulated NES game.
The most difficult challenge we faced was implementing multithreading into our hack. Since the
pyautogui stuff would be holding down keyboard inputs for set amounts of time, we knew it would need to be its own subprocess. Initially, we tried to make the process an asynchronous call from our main MIDI parsing function, but that proved to be extremely confusing. Our next approach was to have the two processes as completely separate (launched separately by the command line), but communicating with each other through a
pickled file. This also was a troublesome approach as file I/O was difficult to deal with. Finally, we ended up using the
queue libraries to allow for easy communication between the two processes. This proved to be succesful.
The Piano GUI
The final element was the keyboard GUI that indicated to the user what notes on the piano they needed to press in order to play the melody. We conveniently used
pygame to draw this UI. We also took advantage of the fact that the piano is a 12 note repeating scale to do some modular arithmetic and calculate where to draw the indicating rectangles.