Melody Mentor

Inspiration

Music education is highly beneficial to young students, boosting academic performance, creativity, confidence, emotional stability, and mental health. However, access to music education has been steadily declining since 1977 (National Assessment of Educational Progress in the Arts). A USC study recently found that learning an instrument enhances cognitive function, increases neural connections, and improves overall wellbeing — especially important during today's youth mental health crisis. Research from the NAMM Foundation also shows that schools with strong music programs have 18% higher graduation rates — yet funding for these programs continues to shrink. With less funding and support, young musicians often struggle and quit when they don't get the guidance they need. So, we wanted to build a tool that helps students by giving them the personalized support to keep growing and improving instead of feeling frustrated and giving up. Allowing more students to continue a music education and reap its benefits.

What it does

Melody Mentor is a personalized music assistant that helps young musicians develop their musical skills.

Users can:

🎵 Upload or record videos of themselves playing
🎼 Add their sheet music for deeper analysis
🤖 Receive individualized feedback on their performance
💬 Chat directly with Melody Mentor for specific advice

Our goal is to supplement music education and make personalized guidance accessible to every young musician, keeping them motivated and excited to learn.

How we built it

-frontend The frontend of Melody Mentor is built with React. It includes functions that handle uploads, session starts, webcam recording, and submissions to the AI system. It keeps track of unique user logins.

-backend The backend is written in Python and hosted using Flask. Our database is MongoDB, which stores user profiles and session histories. We integrated Google Gemini to process user-submitted videos, music, and questions, providing personalized feedback and suggestions. After each chat session, Gemini generates a summary, which we store in MongoDB along with the user's history. This allows future interactions to be more personalized and informed by previous sessions.

Challenges We Ran Into

CORS (Cross-Origin Resource Sharing):
We encountered CORS issues when trying to connect our React frontend with the Python (Flask) backend. Resolving this required significant troubleshooting to properly configure headers and permissions, ensuring secure communication between the frontend and backend.
Live Recording:
Recording live video, saving it, and sending it to the backend presented several challenges. We struggled with figuring out how to capture, format, and transmit the data correctly. After much trial and error, we realized that we needed to convert the recording into a file, send it to the backend, and then save it as a temporary file on the server.

Accomplishments We're Proud Of

This was the first hackathon for several members of our team, so we’re incredibly proud to have completed our full-stack project together. We successfully integrated the frontend and backend, tackled complex challenges like CORS and live video recording, and delivered a fully functional prototype. The experience was both rewarding and a great learning opportunity for the whole team.

What's next for Melody Mentor

We’re planning to add the ability for live feedback during practice sessions, allowing users to receive real-time suggestions and corrections as they play. Additionally, we aim to implement practice streaks to encourage user engagement, motivating them to keep practicing and improving their skills consistently.

We're excited to keep developing Melody Mentor so every student can have the support they need to keep making music! 🎶