Inspiration
Sound is all around us, yet we never actually get to see how it travels. Every conversation, piece of music, or everyday noise sends sound waves through our environment, bouncing off walls, interacting with objects, and gradually losing energy.
We wanted to answer a simple question: What if we could see sound?
Inspired by the hackathon theme "New Worlds", we decided to explore a world that already exists around us but is invisible to the human eye.
Our idea was to create an interactive 3D acoustic simulator that transforms sound propagation into something visual. Rather than building a traditional music visualiser, we wanted to demonstrate the physics behind sound, allowing users to see how the environment itself influences what we hear.
This led us to create SONOSCAPE, an interactive acoustic ray-tracing simulator that reveals the invisible world of sound.
What it does
SONOSCAPE allows users to visualise and explore how sound travels through three-dimensional environments.
Sound sources emit thousands of acoustic rays that travel through a virtual room, reflect off surfaces, and lose energy depending on the materials they encounter. These rays are rendered as glowing lines, with their brightness representing the remaining acoustic energy.
Users can explore different environments, ranging from simple rectangular rooms to a furnished office, and experiment with how different layouts and materials affect sound propagation.
Some of SONOSCAPE's main features include:
- Real-time acoustic ray tracing: Simulates sound propagation, reflections, and material-dependent energy absorption.
- Multiple visualisation modes: Switch between individual acoustic rays, approximate wavefronts, and a combination of both.
- Dynamic ray sampling: Generates new ray directions to improve spherical coverage and visual smoothness.
- Temporal smoothing: Reduces visual flickering, with adjustable settings that can be enabled or disabled.
- Interactive environments: Explore different room layouts, including a realistic office with desks, chairs, monitors, and other furniture.
- Environment presets: Compare reflective and absorbent environments with different acoustic configurations.
- Multiple sound sources: Visualise different sounds originating from different positions within a room.
- MP3 integration: Import music and watch the changing audio amplitudes influence the visualisation.
- Acoustic Time Explorer: Pause, rewind, slow down, and advance sound propagation millisecond by millisecond.
- Customisable simulation: Adjust ray counts, maximum reflections, simulation duration, FPS, and other settings.
- Interactive camera: Rotate around the room, adjust camera height, and change the field of view.
One of the most interesting features is the ability to pause sound in time.
Sound travels at approximately 343 metres per second, meaning it can cross an ordinary room in just a few milliseconds. SONOSCAPE allows users to slow down this process, observe reflections as they happen, and explore how acoustic energy moves through the environment.
How we built it
We developed SONOSCAPE using C++20, with raylib and OpenGL for rendering the 3D environment, visualising acoustic propagation, and handling user interaction.
We used CMake to manage compilation and built a custom CPU-based acoustic ray tracer to simulate sound travelling through the environment.
Each sound source emits rays distributed approximately uniformly across a sphere. When a ray hits a surface, we calculate its reflected direction using:
$$ \mathbf{R}=\mathbf{D}-2(\mathbf{D}\cdot\mathbf{N})\mathbf{N} $$
Where (\mathbf{D}) is the incoming direction and (\mathbf{N}) is the surface normal.
We then calculate the remaining acoustic energy after reflection using:
$$ E_{\text{reflected}}=E_{\text{incident}}(1-\alpha) $$
Where (\alpha) is the material's absorption coefficient.
This allows different materials, such as concrete, wood, carpet, and acoustic foam, to influence the visualised reflections.
To maintain performance, we separated the acoustic calculations from the rendering system. We implemented ray-path caching, dynamic directional sampling, and progressive accumulation to improve visual density without unnecessarily repeating calculations.
We also introduced configurable temporal smoothing to help produce a more continuous-looking representation of sound propagation.
For audio integration, we implemented MP3 playback and analysed the audio signal using root-mean-square (RMS) amplitude. These amplitude values control the visual intensity of the sound rays, allowing music to influence how the room is illuminated.
One of our most important architectural decisions was to separate simulation time from rendering time. This allowed us to implement slow-motion propagation, precise time stepping, and timeline scrubbing independently of the application's FPS.
The application was developed using C++20, raylib, OpenGL, CMake, Git, and AI-assisted coding tools including ChatGPT Codex.
Challenges we ran into
One of our biggest challenges was balancing simulation accuracy, performance, and visual clarity.
Simulating sound accurately is significantly more complicated than simply drawing rays. Real sound exhibits wave behaviours such as diffraction and interference, which would be computationally expensive to simulate accurately in real time.
We chose a geometric-acoustics approach that allowed us to model reflections and material absorption while maintaining interactive performance.
Another challenge was making thousands of acoustic rays look visually smooth. Initially, rays followed fixed paths, which could create noticeable gaps and make the wavefront appear fragmented.
We addressed this by introducing dynamic ray sampling, progressive accumulation, and configurable temporal smoothing.
Timing was another major challenge. Sound travels so quickly that reflections happen almost instantaneously at normal playback speeds. We had to design a separate simulation clock to slow down propagation and allow users to examine individual events.
Expanding the original room into a furnished office also introduced complications. We needed the furniture to look recognisable while maintaining simplified collision geometry for efficient ray tracing.
Finally, integrating MP3 playback required us to manage audio decoding, amplitude extraction, playback states, and synchronisation with the visualisation. We encountered issues with replaying audio after it finished, which required improvements to the playback-state handling.
Each of these challenges forced us to think carefully about how to balance technical accuracy with an understandable and enjoyable user experience.
Accomplishments that we're proud of
Our biggest accomplishment is turning an ambitious idea into a functioning, interactive 3D acoustic simulator within the constraints of a university hackathon.
We're particularly proud of:
- Building our own acoustic ray tracer in C++, rather than relying on a complete third-party acoustic simulation engine.
- Achieving high rendering performance, with the simulator handling tens of thousands of acoustic rays while remaining interactive.
- Implementing the Acoustic Time Explorer, which lets users pause and step through sound propagation in milliseconds.
- Creating multiple visualisation modes, allowing users to explore both individual ray paths and approximate acoustic wavefronts.
- Developing configurable dynamic sampling and temporal smoothing, improving visual clarity while preserving the underlying ray calculations.
- Building a realistic office environment, where furniture, partitions, and surfaces influence acoustic propagation.
- Integrating real audio, allowing users to import MP3 files and visualise their changing amplitudes.
- Creating a highly customisable simulation, giving users control over ray counts, reflection limits, rendering performance, materials, and propagation time.
We're also proud that SONOSCAPE isn't simply a visual effect. It uses real geometric calculations to determine where sound rays travel, what they hit, how they reflect, and how much energy remains.
Seeing the result come together as a complete interactive application was incredibly rewarding.
What we learned
Building SONOSCAPE taught us a great deal about acoustics, computer graphics, and software development.
We gained a deeper understanding of how sound propagates through environments, how surfaces affect acoustic energy, and how ray tracing can be used to approximate complex physical phenomena.
We also learned about 3D vector mathematics, geometric intersections, reflection calculations, audio signal processing, and real-time rendering.
Developing the simulation made us appreciate the importance of separating physical calculations from visual representation. Improving how something looks does not necessarily mean improving the accuracy of the simulation, and maintaining that distinction was essential.
We also learned how important performance optimisation becomes when dealing with thousands of ray intersections and dynamically changing visualisations.
Perhaps our biggest takeaway was that making something visible isn't the same as making it understandable.
We had to think about how users would interpret what they were seeing. Features such as slow-motion propagation, material comparisons, and different visualisation modes made the underlying physics much easier to explore.
Finally, we gained practical experience integrating different systems into a larger C++ application and using AI-assisted development tools to accelerate implementation and debugging.
What's next for We'reWhackingIt
We believe SONOSCAPE has potential beyond the hackathon.
One of our main goals is to introduce frequency-dependent acoustic simulation. Currently, our simplified material model uses general absorption coefficients, but real materials absorb different frequencies by different amounts. Modelling this would make the simulation more realistic.
We would also like to explore more advanced acoustic physics, including diffraction, interference, and reverberation.
Another exciting possibility is introducing a virtual listener. Rather than only seeing the sound propagation, users could position a listener in the environment and hear how the room changes the sound.
We also want to support importing custom 3D environments, allowing users to simulate acoustic propagation in classrooms, concert halls, recording studios, and other spaces.
In the longer term, SONOSCAPE could become an educational tool for understanding acoustics or a way to experiment with architectural sound design.
Ultimately, we want to continue developing the idea that inspired the project in the first place:
What if we could explore the world of sound as easily as we explore the world around us?
SONOSCAPE is our first step towards making that possible.