All three of us, as former or current musicians, admit that music has played a large part in our lives and believe that spreading the joy that it brings can definitely be improved and made more immersive for music students everywhere using technology. We want Mixed Musical Reality to be a tool that makes learning all types of instruments fun and instantly responsive in terms of feedback so that students can learn quickly and in an intuitive and game-like environment.

What it does

Mixed Musical Reality puts you in front of virtual instruments that are fully interactive and presents you with songs that you can learn and play through, all while receiving instantaneous feedback with each note you play.

How we built it

We incorporated the cutting edge holographic mixed reality headset HoloLens2 to create this project, and we took full advantage of its hand joint gestures to develop the functionality in our application.

Challenges we ran into

Being that the technology is still so young and we are lucky enough to be the first to develop applications for the system, HoloLens2 definitely presented us with a lot of hurdles along the way in terms of build errors, missing package errors, and the like.

Accomplishments that we're proud of

We're proud that we took a big leap of faith and tried to develop on this platform eventhough none of us had any Unity experience let alone HoloLens experience. It was a great learning experience for all of us.

What we learned

We learned how to work in a team of people who didn't know each other prior to the Hackathon, and we learned the basics of Unity, HoloLens2, and general game development in a relatively short amount of time.

What's next for Mixed Musical Reality

We want to continue iterating on the application to really make it stand out above the crowd in terms of musical education. In particular, we want to implement full hand joint support for playing the piano alongside a ghost hand demonstration prior to your turning playing because we believe this will serve as a good example in terms of piano fingering technique.

Built With

Share this project: