Waverio Studio

Inspiration

When I was in my senior year of high school, my passion for music recording began. Going through a tough patch in my life, I listened to a ton of music as a way to disconnect from reality and feel more grounded. Listening to bigger artists such as Drake, Don Toliver, and Teezo Touchdown showed me how music can reach the hearts of millions around the world, and I was influenced to try music production myself.

When I first started making music, vocal mixing was probably the toughest part. Understanding how to use a DAW (Digital Audio Workstation) felt like operating a plane, I was bombarded by knobs, sliders, and controls, and I had no idea where to even get started. Getting my vocals to sound even halfway decent on any song was a complete challenge. A couple of years later, I can now say that I can comfortably use various effects such as autotune, reverb, and equalizers to mix my vocal takes with a beat, and I consider myself somewhat experienced. Understanding all the pain and frustration it took to get to this point, I built Waverio Studio alongside my partner, who has supported me in every way throughout my music journey, as a way for beginner artists to be able to produce music without the hassle of spending months learning vocal mixing, but as a way where they can learn it as they go.

What it does

Waverio Studio is a Digital Audio Workstation where artists are able to place beats, record vocals, and utilize an AI assistant to help with the engineering of their vocals. This AI assistant reads a written description of the sound they want, along with measurements of their recording, and builds a custom preset they can still edit.

How we built it

This project uses a React and TypeScript interface to run on a web browser, and uses libraries such as Tone.js to run the audio locally through the Web Audio API. The browser stores data locally through the use of localStorage and IndexedDB, which are used to store sessions and clip audio.

When a user records an audio clip, that snippet is analyzed in the browser by mixing the audio down to one channel, then cutting it into short overlapping frames of 2,048 samples. These frames are analyzed for different measurements such as loudness, noise floor, peak, crest, pitch, etc. The user can then input a prompt describing what they would like their vocals to sound like. These measurements, along with the prompt, are then sent to the server.

A small Node API is in charge of sending this data over to a Hugging Face model (Qwen/Qwen3-8B), as well as information such as a few example presets and a catalog of available effects. These effects were given as Tone.js nodes, but several others, such as autotune, are either custom made or utilize other libraries (such as Rubber Band). The AI returns a preset (which is a chain of vocal effects) in a JSON format, which is then validated and clamped before being utilized.

Challenges we ran into

One of the biggest challenges that we ran into while developing this project was the overall workflow. Considering that this was our first hackathon, we didn't understand how to go about tackling our problem. One specific issue that we learned how to solve as we went along was the use of GitHub and the creation of a repo. Other challenges we ran into included challenges with the actual architecture, one of which being the AI model not fully understanding how to create the right preset. Providing it with proper seeding and syntax allowed it to be far more effective. Another challenge was the actual implementation of certain effects, such as autotune. The pitch correction in both Tone.js and Rubber Band is very limited, and sounds a lot rougher compared to those used in professional DAWs. This is a problem we still aim to tackle in our future solutions.

Accomplishments that we're proud of, and what we learned

We are proud of the fact that we stayed dedicated to the task, regardless of challenges that popped up. We successfully created a web app where users are able to record or upload vocals, prompt AI, and generate a customized and editable preset. At some points, it felt as if completing this project would be impossible in the time given, however, we pushed through and gave it our all. On the behavioral side, we learned a lot about teamwork, working collaboratively, time management, delegation of tasks, and consistency. On the technical side, we learned about proper prompting of AI, GitHub, creating task specs, and hosting of web apps.

What's next for Waverio Studio

We aim to improve upon this application by exploring potential implementations of better effects, which would overall create better sounding vocal presets. We also aim to explore possible fine-tuning of the model we had utilized in order to provide it with more knowledge of vocal mixing.