MuseEQ Suite

Inspiration

We wanted to spread our passion about the intersection between music, computer science, and mathematics. Specifically, we were interested in digital signal processing for music applications. This involves understanding that instruments can be modelled not only with sound, but with graphs and math equations. We developed an ecosystem of apps to spread our passion for music, math, and computer science to students, educators, scientists, engineers.

What it does

The MuseEQ Suite includes the MuseEQ Digital Audio Workstation (DAW) and the MuseEQ Mobile Music Player.

The MuseEQ DAW allows users to define their own timbres with a math equation and variable amplitude (volume), frequency (pitch), and timestep (beat). Users can also view a sample graph of what their timbre equation looks like. Finally, users can publish their musical compositions for others to listen to using the MuseEQ Mobile Music Player.

How we built it

We divided tasks between the DAW and the music player.

The DAW utilizes GTK for creating a desktop GUI. Multithreading was used in order to minimize buffering times. This involved partitioning large two-dimensional arrays with NumPy. To parse a user-defined timbre equation, we reviewed and implemented the Shunting Yard algorithm in Python. Since GTK supported the integration of MatplotLib, we visualized timbres with MatplotLib. Audio playback and file outut required SciPy and SimpleAudio, respectively. AppWrite was used to post audio files to a server backend.

The mobile music player utilizes Appwrite as well, though to get audio data from the same backend. React-Native was utilized to develop the mobile app. React-Native-Track-Player played a key role in supporting the playback of audio.

Challenges we ran into

One of the greatest challenges was overcoming the long runtimes associated with converting user-defined equations into audio serially. As a result, we had to learn how to implement multithreading for this project. Also, there were a lot of UI components that had to be managed on the DAW, such as the piano roll, graphs, and task bar buttons. In addition, this was our first time interacting with a no-code backend solution such as AppWrite. The mobile app also was a challenge to develope, as audio playback requeres an understanding about asyncronous programming. Overall, this was our first experience writing a mobile app.

Accomplishments that we're proud of

We are proud to have been able to write a multithreaded DAW to speed up computions. We are also proud to have been able to implement complext algorithms such as the shunting yard algorithm for parsing. We are also extremely proud to have build a multi-client architecture, which in this cas consists of a DAW posting audio files, and a music player getting audio files. In addition, this was our first time making a large project that involved working with audio files. We also learned about React-Native mobile app development and using a no-code backend solution such as Appwrite.

What we learned

We learned about multi-threaded computing, React-Native mobile development, parsing algorithms, Numpy, SciPi, WAV file IO libraries building a multi-client architecture, and working with a no-code backend.

What's next for MuseEQ Suite

We hope to further develop the app to have proper users, artists, and fully flegded ecosystem for user generated music. We also plan on optimizing the DAWs IO speed.