-
-
Homepage
-
Choosing a Lesson
-
Practice Session
-
A simplified model of the Audio Processing Script
-
Our Logo!
🎸 Learning Guitar through AR
Learning an instrument can be a really tough challenge. As guitar players ourselves, we thought back to struggling during practice, and we also acknowledged how intuitive games like Guitar Hero were, making the act of playing guitar cool and fun. GuitAR brings this to real guitar players, allowing a more approachable and convenient method for anyone looking to learn guitar.
💡 What it does
GuitAR is an augmented reality application which assists users with learning and practicing guitar. GuitAR has sets of lessons and play-along songs, where users are guided to play the notes of the song through a visualization of the guitar fret-board. GuitAR will then listen for the user to play the correct note on the guitar, which will allow them to progress through the song.
🛠️ How we built it
We developed an AR application with a set of lessons and songs, which were stored using a custom data type that represented the fret-board with binary values. This allowed us to easily display the note a user needed to play at a given moment. We also mapped the custom note data to their corresponding note names. Our application was also backed by an audio processing algorithm, which utilized a spectrogram to identify frequencies, as well as manual implementations of algorithms for further audio processing and filtering. The algorithm returns the identifying name of the note, which we validated against the note that was pending play by the user. To further enhance user experience, we also allowed our augmented reality objects to be spatially configurable, allowing the user more comfort in their virtual environment by letting them move objects around as they please.
🧩 Challenges we ran into
The audio processing algorithm was a beast of a task. None of us had done audio processing before, so we researched some libraries to find how to make this kind of algorithm. We originally wanted to make a real-time polyphonic pitch detector, but quickly found that the Quest 3 being based off Android, and Unity being based in C#, limited our options for the integration of pre-existing libraries. We spent a lot of time chasing shadows before finally pivoting and implementing a version of the YIN Algorithm. Seeing as audio processing is the lynchpin for a guitar-teaching app, a lot was riding on this algorithm working and working well, and thankfully, we were able to get it where it needed to be in time. Unity development was also a challenge as most of us had never touched Unity before this.
🎉 Accomplishments that we're proud of
For 3 out of 4 members in our group, this was our first time using Unity for anything, and we're all very proud of getting acclimated to the tool very quickly. We're all very proud of the audio processing algorithm, as well as the way we represented the frets using binary data. We're finally very proud of coming up with a cohesive application in such a short period of time.
📚 What we learned
- Spend more time planning out the thoroughly
- Divide work among members efficiently
- Rest!!!
- Clear communication is key!
- Test tools on our hardware beforehand
- Don't import non-native assets to Unity!!!
🚀 What's next for GuitAR
We have many goals going forward! Some goals include advanced chord detection, adding more songs/lessons and automation the data classifying process, play along to videos, and importing of your own songs.
Log in or sign up for Devpost to join the conversation.