Swara Detector

See the music: real-time swara (musical notes) detection and pitch visualization for Indian classical musicians

Comment

Inspiration

For the inexperienced learner, understanding the difference between different notes is very difficult, and by creating a software that can help identify different notes we are giving learners who may not have access to professional vocal or instrumental training, which develops a trained ear, the opportunity to develop the skills they need to succeed in music.

By bridging the gap between ancient musical tradition and modern technology, we’re making the beautiful complexity of Indian classical music accessible to everyone, regardless of their background, location, or resources. Our mission is to preserve the mathematical precision of this ancient art form while empowering the next generation of musicians with tools for success.

What it does

Swara Detector identifies the frequency/pitch of sounds and labels them according to the sargam system (Sa Re Ga ma Pa Dha Ni Sa scale) based on their proportions, since Indian music uses only one key, which in theory could be any note. It then displays the identified notes.

We created Swara Detector to democratize Indian classical music education. Our software provides real-time visual feedback that helps learners:

  • Identify precise swaras as they sing or play instruments
  • Understand pitch relationships between different notes
  • Develop auditory skills through immediate, objective feedback
  • Practice effectively without constant supervision

This tool is particularly transformative for:

  • Self-taught musicians seeking structured feedback
  • Students in remote areas with limited access to gurus
  • Educational institutions looking to supplement traditional teaching
  • Music enthusiasts exploring Indian classical music for the first time

How we built it

Architecture Overview Swara Detector is built as a client-side React application that processes audio entirely in the browser, ensuring low latency and privacy since no audio data leaves the user's device.

Tech Stack

  • React - Frontend framework
  • Web Audio API - Real-time audio processing
  • CSS3 - Styling and responsive design
  • JavaScript (ES6+) - Application logic
  • Vercel - Deployment platform

Challenges we ran into

We struggled to achieve high accuracy in detecting swaras, which was expected given the many variations in notes. A fundamental challenge was that Indian classical music uses relative pitch rather than the fixed frequencies of Western music. While Western notes have standardized frequencies, Indian swaras are entirely relative to the performer's base Sa.

This required us to design a unique system where users input their base frequency, and all other swaras are calculated as mathematical ratios from that foundation. We pushed past these obstacles by leveraging the pitch perception skills of the musicians on our team to refine our detection algorithms and validate our ratio-based approach.

Accomplishments that we're proud of

Technical Breakthroughs We built a real-time pitch detection engine from scratch using Web Audio API, capable of processing live audio with millisecond latency. We faithfully implemented the ancient swara ratio system with modern JavaScript and created a responsive Canvas graph that displays pitch movement with professional-grade performance.

Innovation & Impact We successfully merged centuries-old Indian classical music theory with cutting-edge web technology. Our tool makes advanced musical training accessible to learners at all levels.

Technical Excellence We built our entire signal processing pipeline without relying on external libraries and achieved smooth 60fps visualization while processing complex audio data directly in the browser.

User Experience Achievements We transformed complex signal processing into a beautiful, easy-to-use interface where users can see their pitch accuracy improving in real-time through instant visual feedback.

What Makes This Special We created something that actually works for its intended purpose. Watching the system correctly identify swaras as someone sings, seeing the visual feedback when they hit the right notes, validates our hard work and proves we've built something truly valuable for music education.

From struggling with FFT arrays to achieving flawless swara detection, every breakthrough moment is etched in our memory!

What we learned

Technical Deep Dives We mastered the Web Audio API, progressing from basic audio contexts to complex real-time signal processing pipelines. We learned to implement FFT-based pitch detection and handle the complexities of vocal harmonics. We optimized React performance to manage state updates without blocking the main thread during audio processing, and created smooth Canvas animations that respond instantly to user input.

The Mathematics of Music We discovered why Indian classical music requires precise mathematical ratios rather than equal spacing, learning the ancient relationships that define each note. Through testing, we found that the human voice requires ±5% tolerance for practical swara recognition.

Cross-Disciplinary Insights We bridged music theory and code by translating abstract musical concepts into concrete algorithms. We designed interfaces that teach while they measure, and learned to respect traditional knowledge while building modern tools.

Most Valuable Lesson We learned that the hardest problems often have the most elegant solutions. What started as a complex signal processing challenge became a beautiful intersection of mathematics, music, and human-computer interaction, proving that technology can enhance tradition without diminishing it.

What's next for Swara Detector

Enhanced Raga Intelligence In the future, we plan to add raga recognition using a Markov chain-based machine learning model that analyzes note sequences and patterns. This will enable the system to identify which raga you're practicing in real-time, provide feedback on raga-specific rules and phrasing, and detect common mistakes in raga progression.

Integrated Practice Environment We'll integrate a digital Tanpura and Tabla for a natural pitch reference and rhythmic accompaniment during practice. This includes customizable drone settings with Sa-Pa or Sa-Ma combinations, multiple tempo options for different compositions, and real-time rhythm synchronization to enhance practice sessions.

Share this project:

Updates

posted an update —

As we complete one year since the ideation of this project, here are some updates!

TL;DR

  • Digital Tanpura is here! As promised earlier, I have integrated a fully synthesized, browser-native digital Tanpura into the project. It features traditional 4-string acoustic tuning (Pa–Sa–Sa–Sa and Ma–Sa–Sa–Sa), continuous drone synthesis, and live sync with the pitch detector.

  • Introducing Ragas for Right Now! You can now discover which ragas are traditionally associated with the current time of day (the Prahar system), along with the moods they are believed to evoke, and listen to classical renditions with a single click.

Under the Hood (For the Tech & Music Geeks)

For those curious about how this was built without relying on heavy recorded audio samples or a backend server:

  • Pure Browser Audio Synthesis (Zero Samples): The Tanpura runs entirely on the Web Audio API. Rather than looping static MP3s, every pluck is synthesized in real time using custom Fourier-series PeriodicWave tables that model the rich harmonic overtone profile of steel and bronze strings.

  • Authentic 4-String Acoustic Tuning: Instead of a generic chord, the engine faithfully implements traditional Indian classical tuning: Mandra Pa/Ma (lower 5th/4th at 0.75× or 0.667×) → Jodi Madhya Sa (1.0×) → Jodi Madhya Sa (1.0×) → Kharja Mandra Sa (deep lower root at 0.5×). The Kharja string is dynamically given extra sustain and acoustic body to ground the singer.

  • Jivari Bridge Physics Simulation: The iconic buzzing shimmer of a Tanpura’s bone bridge and cotton thread (jivari) is simulated using dual detuned oscillators combined with a dynamic resonant BiquadFilterNode sweep on each attack. The detuning uses a proportional pitch multiplier (Δf = f × multiplier) so the acoustic chorus rate scales naturally whether you're singing at 100 Hz bass or 400 Hz soprano.

  • Audio-Clock Lookahead Scheduler: Precise timing is decoupled from JavaScript's main thread and setInterval jitter. A lightweight lookahead scheduler queues plucks directly onto the Web Audio hardware clock (audioCtx.currentTime), allowing live tempo, volume, and tonic changes without gaps or audible clicks.

  • The 8-Prahar Time Engine: "Ragas for Right Now" divides the 24-hour day into the 8 traditional Indian classical Prahars (including late-night midnight wrapping), algorithmically matching your local clock to 30 curated ragas and their emotional rasas.

With the everlasting grace of the ever-compassionate Maa Saraswati, I have been fortunate enough to learn so much about classical music over the past year. The project originally started as an effort to develop a system that could detect ragas. So, it would not be wrong to say that even though the project is called Swara Detector, the raga remains its ultimate focus.

Over the last year, I embarked on a journey to explore and understand ragas. And if you know me well enough, it might surprise you just how much I discovered.

To begin with, we know that ragas can profoundly influence our moods and emotional experiences. I wanted to explore this for myself, and I genuinely experienced it firsthand while experimenting.

To be frank, my primary motivation was to explore whether classical music could help people struggling with mental health challenges. This eventually led me to integrate the concept of Raga Chikitsa (Music Therapy) into one of my other projects, ManasCare.

Of course, I am still learning, and I would love to explore this connection more rigorously. There is a lot to understand about the relationship between music, the mind, and well-being, and I believe this is an area worth investigating.

Additionally, I discovered something about ragas that I had never encountered before.

If we are close, you might know that I am quite enthusiastic about astrology, as you may have guessed from another one of my projects, AstroBot.

During my exploration, I came across the concept of using ragas to influence planetary energies in Vedic astrology. Some spiritual interpretations connect specific ragas with the body’s subtle energy centers, or chakras, and associate them with particular planetary qualities.

I started experimenting with these ideas myself, and the experiences were fascinating. Of course, I would like to understand these connections much more deeply before drawing any definitive conclusions.

But why am I sharing all this?

Because these discoveries have motivated me to continue working on Swara Detector, even though I still have a lot to learn about the domain. I might hopefully explore all these concepts in the future updates of Swara Detector.

Ultimately, I want this project to have the potential to benefit people. It can certainly help aspiring musicians learn and explore Indian classical music, but could it also contribute to our understanding of music’s relationship with mental well-being?

I genuinely hope so.

I know there is a long way to go, and I do not want to make promises I cannot yet substantiate. Nevertheless, the possibility alone gives me a reason to keep exploring.

Another source of motivation has been the surprisingly encouraging analytics this project has received so far. Knowing that people are finding it interesting makes me want to continue building it.

When will you hear from the Swara Detector again?

Right now, I am working on what is easily one of the most complex projects I have ever undertaken. Swara Detector deserves my undivided attention, and I would rather take my time than rush out something that does not meet my expectations.

Lastly, I am incredibly grateful to have had the opportunity to explore this art form. I never imagined I would become so interested in it, but here we are!

I dare say Indian classical music is the language of the Gods.

Or perhaps the language of the Gods comes second to mathematics. ;)

And now, for a little personal update.

If you have not guessed by now, I am trying to take up a new hobby.

That hobby is writing.

Shameless plug: I will be writing about all sorts of things here, mostly nerdy academic stuff: writing.sumitk.me.

I might also write in greater detail about some of my discoveries while exploring music, including the fascinating relationship between music and Fibonacci numbers. That last one is probably my favorite.

Thank you for reading!

And please, share your feedback on Swara Detector. I really need it!

Log in or sign up for Devpost to join the conversation.

Submission history