HarmonAI

Inspiration

Many people have a fear of singing because they're afraid of embarrassing themselves. We also know that singing lessons can be pricey, time-consuming, and intimidating. That is why we wanted to build something practical, and to help others who want to express themselves and share the same fear that we do.

What it does

HarmonAI acts as a personal vocal coach that lets users record themselves and then provides a detailed report of their singing voice along with an analysis, tips, and exercises to improve. Users are able to do a general analysis by singing a simple harmony or scale, or compare themselves to a song to learn how to sing more like their favourite artists.

How we built it

We used the Gemini API to power the analysis and comparison of the platform. Gemini also generates a 7-day vocal plan for the user based on their report. We also used Vue for our front-end to combine and route components, and Express as our backend to send HTTP requests between both sides.

Challenges we ran into

Working around the time constraint was our biggest challenge. We had a lot of ideas and features that we wanted to implement, but knew that we could not get to all of it. Additionally, we were all new to using Gemini, so figuring out how to incorporate it into a program was a difficult ask. There was also the learning curve of using Gemini to help us code. While it was a steep learning curve at first, it definitely helped us ship our project with more features than we could have thought in the 12-hour timeframe we were given.

Accomplishments that we're proud of

We're proud that we were able to develop a complete and useful tool that we are happy to use and share. Our idea has so much room for advancement and growth, and with the newfound skills that we gained from working on this, we could easily envision ourselves shipping more features to this project.

What we learned

With most of us being new to hackathons, we learned a lot about project development and collaboration. Most importantly, we learned a lot about using AI tools through working with the Gemini API. We learned that not only could we use text to prompt it, but we could also upload files to have the API interpret and provide a response for it. We were also amazed by how emotional its analysis could be. It was able to tell qualitative features of a voice like timbre and texture through an audio file.

What's next for HarmonAI

As mentioned previously, there is still much more we want to add. Implementing a vocal range analysis will allow users to not only sing better, but also figure out what other pieces can fit them. We also want to include other ways of gamifying the process to make the experience less daunting. For additional motivation, an improvement roadmap can help users compare and see how far they have come. We hope to become the Duolingo of singing!

Built With

• express.js
• gemini
• typescript
• vite
• vue