Example wave

qSound

qSound is an interactive, over-the-top Python project that translates live music into a surreal, quantum-infused visual experience. It uses real-time audio capture to control a quantum simulation, rendering qubits as waves and particles that shift and swirl in response to every beat.

Overview

qSound captures real-time audio input and breaks it down into essential elements like frequency, amplitude, and phase. These sound properties are mapped onto quantum states, visualizing them as vibrant, animated waves. The result is an audiovisual experience for anyone interested in the intersection of sound and quantum mechanics or just looking for something cool and experimental.

Features

Real-Time Audio Processing: Captures and processes live audio input with customizable device selection.
Quantum Simulation and Visualization: Converts sound parameters into quantum states and displays them in 3D.
Particle-Based Visualization: Generates particles that respond to music intensity, with dynamic speed, color, and movement patterns.
Neural Network Integration: Analyzes musical intensity over time to produce unique, music-driven visualizations.
OpenGL Rendering: Provides real-time rendering for quantum transformations and particle effects.

Inspiration

In the immortal words of Matthew J. Hale Rattigan, "This weekend is about doing things not because we should, but because we can." qSound exists primarily because it allows us to merge the elegance of quantum mechanics with the power of neural networks, transforming sound into art in a complex yet captivating way. As enthusiasts of neural networks and quantum computing, we set out to create an algorithm that doesn’t just interpret sound—it immerses you in an over-engineered spectacle of qubit rotations and particle effects, revealing the hidden beauty of quantum states as they respond to music.

How We Built It

qSound revolves around a single, central function:

Audio Input: Listens to the device’s sound output, sampling attributes like frequency, amplitude, and phase at 16ms intervals, holding up to 2 seconds of data.
Neural Network Analysis: A neural network trained on our custom dataset of 60 song samples returns a music intensity value from -1.0 to 1.0, giving a more accurate tone analysis than larger public datasets.
Quantum Mapping: Maps audio parameters to quantum transformations:
- Amplitude -> Controls qubit rotation strength.
- Frequency -> Adjusts oscillation rates for qubits.
- Phase -> Applies phase shifts to affect qubit orientation. Twelve qubits in paired basis states are rotated based on sound parameters. The result is constantly computed on a separate thread for smooth real-time rendering.
Particle Effects: Particle colors, movement, and spread respond to music intensity, with faster movements and warmer colors for high intensity and cooler, slower particles for softer tones.

Challenges

From start to finish, we encountered several challenges:

Real-Time Visual Updates: Struggling to find libraries with accurate and fast enough real-time audio monitoring and processing required us to create custom extractions from Fast Fourier Transforms
Music Intensity Analysis: Training a neural network to reliably assess music intensity proved trickier than expected but ultimately led to a custom model that outperformed public datasets.
Qubit Mapping Algorithm: Developing a sound-to-qubit mapping algorithm that produced visually compelling results involved extensive experimentation.

Accomplishments

We’re proud of taking an abstract idea – "What if quantum states could be controlled with like, music?" – and making it a fully functional algorithm in just 36 hours.

What We Learned

We delved into new territories with OpenGL, audio monitoring, and quantum state mapping. Every step introduced a new learning curve, but each brought us closer to our final vision.

What's Next for qSound

Project it on a screen, throw on your favorite song, and enjoy the show! (would be interested to see the feasability of attemping to reverse the transformations, as a fun follow-up)