• 

• 

• 

• 

• 

• 

• 

EccoSampler

EccoSampler transforms ordinary recordings into evolving, editable instruments for real-time experimental sound design.

Inspiration

To preface, I am not a software developer. I studied biology with a minor in data science, and I am about to begin medical school. I have some experience in Python, and worked on a neural cellular automata model at the Lawrence Livermore National Laboratory. Before EccoSampler, I would have never dreamed of building a native audio application or a real-time audio engine. Today, Codex and GPT 5.6, along with general improvement in coding agents, makes this possible (and closer to 5GL in a way)!

My role in the project was as an outsider to software development. I consider myself a student, experimental musician, and holistic thinker. I love music, and most of my coding with Codex is related to music/audio projects. It has, by far, lifted many of the "writer's blocks" that I had, and I am grateful to OpenAI and the many scientists, whom we stand upon the shoulders of, for the paradigm change.

I wanted to make eccojam-style transformations from my own samples without having to continuously use Audacity by hand. Audacity, while a well-iterated program, can loop a selection during playback, but that workflow did not always give me the result I wanted. I had to record my sessions on Audacity with OBS, which is not necessarily ideal for replicating initial conditions if I wanted to recreate it as a later song project. Also, exploring my flashcore ideas in a conventional DAW requires a large collection of specialized plug-ins costing hundreds of dollars (see PolyNodes and other plugins by sonicLAB).

I could not find one accessible interface that combined immediate looping and transformation, real-time playback, parameters, micro-sound design, and exporting features. So, with the help of Codex, I built EccoSampler.

I try my best to listen to all the music that's out there. In the process, I have discovered many interesting genres that don't get enough love. This includes: Eccojams, particularly Daniel Lopatin’s work as Oneohtrix Point Never, Chuck Person, and sunsetcorp, in works like Replica and Memory Vague. Flashcore, an avant-garde genre with roots in speedcore, also caught my attention—especially Laurent Mialon’s work as La Peste and Krystal Jesus. The genre utilizes specialized systems for sounds that conventional tools do not make easily. Stochastic and graphical composition, especially through the works of Iannis Xenakis and the UPIC, suggested an alternative form of musical thinking that does not incorporate conventional notation.

EccoSampler brings those ideas into one approachable instrument with its own musical and visual identity.

What it does

EccoSampler is a native, local-first loop-collage instrument. A user can import one or more recordings and inspect their waveform, tempo, downbeats, sample-level values, and candidate loop seams.

The user can then choose one of five immediate transformation modes (with the help of Codex to describe each preset so you have a better idea):

• Ecco Memory — long phrase loops, recurrence, darkening, and imperfect return
• Vapor Drift — slowed movement, reverse fragments, tape motion, and diffuse space
• Prism Fracture — asymmetric, multi-scale slicing and high-definition collage
• Flash Lattice — dense micro-regions, abrupt rate changes, omissions, folds, and displaced echoes
• Broken Transmission — evolving dropout, leakage, resynchronization, damage, and temporary clean recovery

The modes are shapable through six parameters inspired by the Transform pad in Alchemy (Logic Pro): Density, Instability, Fragment Size, Recurrence, Destruction, and Recall. EccoSampler expands its decisions into an editable arrangement view, akin to Ableton's dual-view system, as a more traditional DAW experience.

The Perform workspace is where I work most closely with; it provides audio-reactive and state-based views, including Signal Field, Graphscore, Kinetic Atrium, and Memory Constellation 3D (inspired by flashcore plugins)—without visual animation misrepresenting the audio engine. Graphscore (akin to Xenakis' UPIC) also has a working instrument workspace where a user can draw a freehand curve, transform the object, and hear the deterministic recompilation.

How I built it

Most of the Build Week version came together during an intense period of roughly three days. I found out about this hackathon a bit later through email, so I had to spend a lot of concentrated effort into this project. Because I have little conventional coding experience and do not personally know how to code in Rust, I used Codex as my main coding buddy.

I supplied my vision for musical direction and product requirements, and required that Codex reference research, test recordings, provide acceptance criteria, and consistently interpret audio and visual feedback. Codex then translated specifications into implementation tasks, wrote and reviewed code, added tests, investigated platform behavior, and helped analyze screenshots and runtime failures. Throughout, I launched the application, listened to its output, compared the interface and behavior with my intentions, described failures precisely, and redirected the work via model steering. I observed that Codex is great at adjusting its chain of thought when I observe something that is not correct.

Technically speaking (with the help of Codex to describe this section), the application is a nine-crate Rust workspace separating analysis, core musical and DSP types, audio hosting, rendering, project state, file I/O, the command-line interface, the GUI, and an extension SDK. The interface uses egui/eframe; local audio output uses CPAL; decoding uses Symphonia with an optional FFmpeg fallback; bounded communication uses preallocated structures; and TOML documents preserve portable recipes and editable project state. Playback and export both use the same prepared ProjectRenderer architecture. Expensive work—decoding, validation, graph construction, source admission, and renderer preparation—happens away from the audio callback. The callback receives prepared state and writes bounded blocks, while the GUI consumes copied telemetry rather than driving DSP. This makes the product’s technical design follow its artistic principle: the result should remain understandable, revisable, and repeatable. Project operations also fail explicitly when a requested graph, route, parameter binding, or safety budget is unsupported. The program does not silently ignore an advanced feature and pretend that it affected the sound.

The submitted code snapshot has 924 workspace tests passing in its default CPAL configuration. Separate GUI configurations and focused release probes were also tested. These checks cover deterministic rendering, block-partition invariance, project persistence, event history, source-frame accuracy, live renderer revisions, export behavior, callback boundaries, safety limits, failure handling, and platform launch paths. This is build-host evidence from Debian through WSL 2; other operating systems are a future direction for EccoSampler.

As the human reviewer, I noted that Codex wrote much of the implementation and has a deep understanding of DSP and realtime computational behavior. I acknowledge that I don't understand every part of the codebase, but of course, it's nothing that can't be fixed with some open-minded learning. The musical system, product judgment, interpretation, and final decisions remained human-directed by me!

Challenges I ran into

Translating musical language into exact behavior

Terms such as “eccojams,” “flashcore,” or “broken transmission” are relatively long-tail outliers in the distribution of music, so I had to emphasize that Codex needed to research these artists before interpreting them; and a real-time engine needs exact rules. Codex had to translate those ideas into source-frame ranges, event probabilities, state transitions, seeds, histories, DSP parameters, bounded buffers, and controls.

Protecting the real-time audio path

A native audio application cannot allow file access, decoding, graph construction, uncontrolled allocation, or expensive GUI work to enter the callback. Maintaining a strict boundary between preparation, playback, telemetry, and interface work became one of the most technically demanding parts of the build. Codex also had to make failures bounded and visible rather than allowing a stalled backend to hang indefinitely. Performance issues, hangs, and crashes were the stopping point multiple times within the past three days.

Teaching Codex what the GUI actually looked and felt like

Codex could modify GUI code and visually reason with screenshots, but it did not consistently understand hierarchy, spacing, proportions, clipping, aesthetics, or interactions from source code alone. I had to launch the program, capture and inspect the result, explain what looked wrong, and steer the next revision manually.

The process became a loop: implement, run, capture, compare, describe, and revise. It's pretty close to the classic scientific method! Codex’s visual abilities helped, but they did not replace human art direction or usability judgment. Computer use on the app was helpful, but did not fix errors that I mentioned on the GUI. This is something that future iterations of Codex can improve on.

Computer audition

This is something that is understated, but Codex and GPT 5.6 does not have any encoding or ability to natively reason about audio that it is provided. The closest thing is GPT-Live, but that is not a substitute for coding and research. I do hope that OpenAI looks into this, similarly to adjacent Qwen and Nvidia models online at the moment. For now, I have to describe via text what I hear (and Codex does its best with what it's given), and use terminal tools to analyze component features of audio. I believe computer audition would be one step closer to a closed-loop iterative scientific method algorithm.

Keeping the scope honest

The project quickly blew up into graphical composition, stochastic instruments, typed routing, memory graphs, spatial simulations, and performance optimization. Some of those systems have deterministic data models, compilers, audible render paths, and truthful visualizations while still lacking complete end-user editors. Acknowledging the limitations within the short amount of time I had was difficult, but something necessary in order to have a working product.

Platform and packaging reality

Development and runtime testing were performed primarily on Debian through WSLg. The repository includes source launch paths for Linux, Windows, and macOS, but native Windows and macOS compilation, device testing, packaging, signing, and clean-machine verification still require dedicated release passes. At the moment, I was not able to code releases for Windows and MacOS (I also do not have an Apple Silicon Mac machine, although I do have a legacy Intel one which developers are starting to abandon...).

Accomplishments that I'm proud of

I am most proud that, despite not knowing Rust or considering myself a coder, I was able to direct the creation of a substantial native audio system in roughly three days of concentrated Build Week work.

EccoSampler is close to a truthful implementation of a DAW from scratch, which I didn't think would be possible, given the proprietary nature of high-performance DAWs. There are also no mainstream experimental DAWs that I know of aside from Renoise, which is a tracker-based DAW, separate from what EccoSampler was aiming for (although, tracker-style implementations could be a future direction).

Other accomplishments include:

• Five audibly distinct transformations built around one approachable workflow
• Seeded, inspectable generation with exact source-frame provenance and event history
• Shared prepared rendering for playback and export
• Editable arrangements, effect racks, generation history, undo/redo, autosave after the first explicit save, and portable project documents
• Audio-reactive views that consume mathematical and statistical analysis, geometry, and renderer telemetry (not purely decorative like a basic audio visualizer, although MilkDrop-like visualizations are potentially another future direction)
• A working Graphscore drawing and transformation workflow with deterministic recompilation
• An Apache-2.0 codebase with explicit third-party and extension boundaries

What I learned

The largest lesson was that coding agents can make ambitious, niche projects possible for people like me, who have strong domain knowledge but limited implementation experience and time.

I also learned that “not being a coder” does not mean having nothing to contribute. Musical listening, product taste/preference, research, experimental design, test selection, failure/limitation recognition, and the ability to explain exactly why a result feels wrong is all part of the engineering process, not something that Codex can realize by itself. My data-science background helped me reason about this process, even when I could not write the code myself. I also learned that, today, it is more imperative than ever to be critical about the results you see (e.g. model hallucinations, form over function, etc.). I also learned that old-school by-the-book efforts are underrated and need more attention! It's inspiring, honestly, more than discouraging.

I learned that coding agents sometimes struggle with evaluating frame accuracy, persistence, bounded failure, and audio quality. As the Jukebox paper by OpenAI mentioned, we lack significant evaluation criteria for automatic musical interpretation, specifically if a clip is emotionally interesting, has control, has the right musical range, or feels coherent. For now, those questions still require a person to listen, look, compare, and decide.

Finally, I learned that a generative creative tool becomes more useful when it preserves what happened, where it came from, which state produced it, what changed, and whether the result can be recreated. But it cannot be fully trusted without human supervision. It is a direct juxtaposition to the sci-fi rooted fear of AI conquering humanity.

What’s next for EccoSampler

Planned work includes:

• Complete native Windows, macOS, and Linux verification, then produce signed and portable releases
• Refine the Ocean Glass interface (including its visual design) across more window sizes while preserving performance, accessibility, and readable visual hierarchy
• Finish complete creation and editing workflows for Probability Loom, Echo Seed, emergent instruments, Kinetic Atrium, Memory Constellation, advanced routing, and scene mutation
• Expand Graphscore beyond its working curve workflow with more object types, assignments, transformations, and clearer editing feedback
• Add deeper performance capture, resampling, MIDI/MPE and OSC control, and carefully bounded DAW integration
• Develop the extension SDK into a safely hosted user-facing ecosystem without pretending that arbitrary plug-in loading already exists
• Publish tutorials and example projects so someone unfamiliar with eccojams, flashcore, or graphical composition can still begin with one sound and understand the result.

The long-term goal is for EccoSampler to become a serious experimental instrument that remains inviting to curious people. And one final note, thank you for reading this far and looking at my project! :)

P.S. regardless of outcome, just wanted to share a quote I liked to fellow humans, here you go: "Someone is sitting in the shade today because someone planted a tree a long time ago." — Warren Buffett

Built With

• codex
• cpal
• desktop-application
• digital-signal-processing
• egui
• generative-music
• open-source
• openai
• real-time-audio
• rust
• wslg