NotePulse

Inspiration

As singers and music lovers, we wanted to explore whether technology could recognize the notes someone is singing in real time. We were inspired by the idea of making pitch recognition more accessible and interactive. Our goal was to combine hardware, audio processing, and machine learning to create a tool that helps singers better understand their voices.

What it does

NotePulse listens to a singer through a microphone and identifies the musical note they are singing in real time. The detected note is then displayed on screen, allowing singers to immediately see what pitch they are producing. This creates a simple and interactive way for singers and music lovers to visualize their voice.

How we built it

We used the STMicroelectronics SensorTile.box PRO to stream microphone audio and built the processing system in Python. Our machine learning model uses extracted audio features such as STFT, chroma, and harmonic information to classify musical notes with an SVM, while a separate FFT-based pitch estimator tracks the singer's fundamental frequency. The results are sent through a WebSocket connection to our interactive dashboard, where the detected note, pitch, confidence, and other audio information are displayed in real time.

Challenges we ran into

Getting the SensorTile.box working was one of our biggest challenges, and multiple teams experienced similar issues. It took us about 5 hours to get the device running, especially when trying to get the microphone audio working and save the raw audio. The ST platform was also difficult to work with and did not provide the flexibility and reliability we needed, which was especially challenging during a time-limited hackathon.

Accomplishments that we're proud of

We're proud that we were able to turn our idea into a working real-time vocal note recognition system despite the challenges we faced with the hardware. We successfully streamed audio from the SensorTile.box PRO, processed the audio, and used machine learning to classify musical notes while also tracking vocal pitch. We also built a live dashboard that displays the detected note and pitch information in real time.

What we learned

We learned that building an ML project with hardware involves much more than just training a model. We gained experience working with audio data, extracting meaningful features, and connecting sensor input to a machine learning workflow. We also learned how important troubleshooting, flexibility, and adapting our approach are when working with unfamiliar hardware under a tight deadline.

What's next for NotePulse

We envision expanding NotePulse from a real-time note recognizer into a personalized vocal practice tool. In the future, NotePulse could compare a singer's performance against a reference song, track how many notes they hit correctly, and provide feedback on areas they can improve. We also want to explore a visual interface that could display performance scores, pitch and timing accuracy, and recommend songs based on a singer's skill level.