Being a musician and wanting to find a way to make use of Augmented Reality, I thought that it would be cool to incorporate this fascinating technology into my practice. I was also inspired by existing apps like Piano 3D and AR Pianist. Although these apps have similar purposes with my idea, they tend to have problems and inaccuracies with aligning the keys on the augmented reality with the user's piano keys, which defeats the purpose of these apps. For these reasons, I decided to try to make my own Augmented Reality piano app using echoAR.

What it does

My app superimposes the video of the piece being played onto the top of the user's keyboard in augmented reality. This is done by recognizing the QR code which is placed behind the piano. The purpose of using the AR is to make learning songs and techniques easier on the piano, since users can see what they need to play above the keyboard.

How I built it

I built this using the echoAR web console and Unity.

Challenges I ran into

One of the biggest challenges was making sure that the keys in the video in the AR lined up with the keys on my keyboard. This aspect is very important, since it is supposed to help the user learn more easily, and is the main purpose behind my idea.

Accomplishments that we're proud of

I'm proud that the AR ended up working out. Being able to work in Unity was also a great accomplishment, since I've never had experience with Unity before this hackathon.

What we learned

I learned a lot about how AR works during this hackathon. Also, working and experimenting with echoAR was a lot of fun, seeing different applications of AR and VR, as well as how to manage 3D content without any experience using echoAR. As a visual learner, working with augmented reality is a great experience.

What's next for Piano: Augmented

I would need to create a create a full app, with a UI, add more piano pieces, and improve the overall stability of the augmented reality.

Share this project: