pianEAR

A webapp that helps musicians unlock their full potential. Our app uses ML to compare sheet music with the notes played by the user.

Inspiration 💡

Practicing something all day, just to be told it's all wrong is not a good feeling. At PianEAR, we know that feeling all too well. As musicians ourselves, it's hard to tell between a B and B flat. At least for beginners. Without a trained ear, it's difficult to tell the difference between notes close together on the scale, making the beginning musician prone to mistakes.

What It Does 📷

This is why we created PianEAR, an artificially intelligent piano teacher, who will catch your mistakes the second you make them. Through this technology, you can improve your piano skills by receiving feedback in real time for each note you play. So no more practicing wrong! A user can go on the website and opt to use our service. As you can see, we will require users to upload a pdf file of their music so we can turn it into a legible array. Next, the user can simply start playing like so… and when they make a mistake it will show through the red font on the screen, all provided in real-time feedback. At the end of the performance, our software will provide a final accuracy rate for the user.

How We Built It 🛠

We have made a website using React, next.js, typescript and tailwind CSS for the consumer product. As well as using other tools, such as Figma for prototyping.

Challenges We Ran Into 📉

One big challenge we ran into during this hackathon was a conflict of interest amongst team members. While we were able to move past it quickly, it did slow us down. Another challenge we faced was not having enough experienced coders on our team. Instead of being able to combine many coder's knowledge, often done in hackathons, we had to be very thoughtful with what we knew and how we applied it. For example, when looking at converting audio to keys, we connected that knowledge to past experience, robotics and how joysticks actually worked.

Accomplishments That We're Proud Of 💪

At PianEAR, we are most proud of the product we've built. As this issue is close to our hearts - and one we experience on a daily basis - our greatest accomplishment was going from the ideation phase to the creation phase, all the way to building it out. While the project is still in its infancy, we are certain the future of PiaEar is bright. Additionally, throughout the challenge, we optimized to gain knowledge in areas we weren't particularly strong in.

What We Learned 🧠

Pre-planning is key to being successful during a hackathon. Spending the time to create an organized schedule and timeline is an important step in setting yourself up for success. Doing so helps your team stay on track and prevents the last-minute rush to get things done. This time, my team and I created a schedule and diligently followed it throughout the event. This allowed us to enjoy the activities and events, along with giving us plenty of time to hack.

What's Next For PianEAR 🔜

Moving forward, PianEAR will continuously improve its accuracy by using Machine Learning to get smarter. But our vision for PianEAR doesn't stop there. We hope to go beyond the current phase of notifying the user when they are incorrect and providing them with the appropriate corrections and suggestions. By doing so, not only will the user know they are wrong, but rather than using trial and error to solve the error, they can simply refer to the web app.