There is one problem that causes rhythm games like Guitar Hero to not be popular today. That is the criticism that states that you do not play 'real' music, but instead only use a guitar-shaped controller. We set out to make a system that will allow the user to learn and play real music.
How it works
This works by using a couple of parts. The main brain of the operation is the Pixy camera, a camera that is configured to find blocks of colors, and output the position. Using this, and a system to convert Midi files into bars similar to how guitar hero is displayed. This Midi file would then be overlaid on top of the video from the Pixy, which would then be fed into a form of VR headset similar to the oculus rift.
Challenges I ran into
We had some issues when it came to communicating with the Pixy and Arduino. We went through the Edison, Leonardo, and the Uno, until the Uno was able to do what we needed. We also learned that we can not pull both coordinates and Video from the Pixy simultaneously, so in order to make a fully functioning camera, we would need two separate cameras.
Accomplishments that I'm proud of
We were able to figure out the communication tricks we needed in order to get coordinates from the Pixy. We also were able to manipulate a MIDI file to display falling notes on an image of a piano.
What I learned
Communication through Arduino boards. Working with Pixy camera. Communicating with mentors and finding help from the right people; making connections.
What's next for Pixy Beats
Integrating a second camera in order to make full use of our project. In addition, optimizing our project in order to make it be able to display the data live. Using Google Cardboard or some other virtual reality project to bring the screen from the computer to your face.