• 

  VibeSeq - main

VibeSeq — Compose Music With Words, Then Shape Every Part

Inspiration

I am a musician and a performer, but I cannot play every instrument equally well.

Most of my songs begin as something small: a melody I sing into a microphone, a short instrumental phrase, a rhythm, or a rough recording made before the idea disappears. The difficult part comes next. To turn that fragment into the smallest arrangement complete enough to share with someone else, I often need instruments that I cannot perform convincingly myself.

I became tired of interrupting that moment to search through Splice. I would browse loops and samples, auditioning one file after another, trying to find something close to the sound I could already hear in my head. By the time I found it, the original emotion and momentum were often gone.

Suno represented the opposite extreme. It could generate an impressive finished song, but the result often felt as though it had crossed the boundary of what I could honestly say I had made. I did not want AI to complete the creative process for me. I wanted it to help me continue the process I had already started.

That tension became the reason for VibeSeq.

I did not want AI to finish the song for me. I wanted it to help me keep playing.

VibeSeq uses language to bridge the instruments I can play and the sounds I can imagine. Instead of generating one finished song, it creates separate musical parts: a drum groove, guitar rhythm, electric piano voicing, bass line, synth phrase, vocal texture, or one-shot transition.

I can describe the part I need, audition variations, choose what belongs, and place it beside my own ideas. The arrangement remains visible, editable, and mine to shape.

VibeSeq explores a different role for generative music: not replacing the musician, but helping a musician carry a fragile idea far enough that another person can finally hear it.

What VibeSeq Does

VibeSeq is a local-first AI music studio built around one continuous workflow:

describe → generate → audition → place → edit → arrange → export

A musician can describe separate drums, guitar, electric piano, bass, synth phrases, transitions, vocal textures, and one-shot sounds. Each result remains an independent musical part rather than being merged into an untouchable finished song.

Tempo-locked loops follow the musical timeline, while one-shot sounds remain fixed events. The arrangement grows cumulatively, so adding guitar means hearing drums and guitar together—not replacing one generated result with another.

When a specific sound needs deeper control, VibeSeq can extract MIDI from selected audio. This is intentionally not the main product story. MIDI extraction is a precision tool: it can turn a tight rhythm into editable notes or reduce a complex synth phrase into a minimal motif without regenerating the entire composition.

Once the required models are installed, generation and transcription run locally. Projects, prompts, seeds, audio relationships, and edits remain on the user’s machine, allowing musicians to continue working without depending on a cloud session or losing their creative context.

How I Built It

VibeSeq combines a TypeScript and React music interface with a local inference service and a Web Audio playback engine.

Stable Audio 3 Medium generates musical parts from natural-language descriptions. MuScriptor Medium can transcribe selected audio into linked, editable MIDI. The application preserves the relationship between the original audio and its MIDI derivative, so conversion never silently destroys the source.

The Studio includes:

• independent Audio and MIDI tracks;
• cumulative arrangement playback;
• waveform and piano-roll editing;
• trim, fade, loop, gain, pan, mute, and solo controls;
• local project persistence and recovery;
• WAV, MIDI, aligned track, and portable project export;
• model, prompt, seed, and source provenance;
• packaged desktop applications for macOS, Windows, and Linux.

VibeSeq was built during OpenAI Build Week with Codex and GPT-5.6. Codex researched musician workflows, translated product requirements into implementation contracts, proposed two distinct UX designs, selected the more user-friendly direction, and built and verified the working application.

Codex also accelerated the local inference integration, audio engine, persistence model, automated tests, desktop packaging, release automation, and the reproducible demo-video pipeline. I defined the product purpose, musical direction, model and licensing boundaries, creative acceptance standards, and final quality decisions.

Challenges

The first major challenge was making generated parts behave like parts of the same song. A collection of individually good sounds is not automatically a coherent arrangement. Key, tempo, meter, loop length, and timing needed to become explicit musical contracts.

For the demonstration, every tonal source follows E natural minor at 120 BPM in 4/4. Drums establish the foundation, guitar and electric piano add harmony, a synth phrase creates motion, one-shots provide transitions, a wordless vocal creates the emotional peak, and bass stabilizes the final arrangement.

The second challenge was maintaining control without making the workflow feel technical. Generation needed to feel immediate, but prompts, variations, model provenance, and source relationships still had to remain visible and understandable.

Local execution created another set of challenges. Large models cannot simply be hidden inside every installer. VibeSeq therefore ships model-free desktop packages, guides users through the appropriate model setup, verifies the required files, and keeps gated model downloads under the user’s control.

Finally, building a real music application required more than displaying waveforms. Playback, editing, persistence, export, model execution, and UI state all needed to agree on the same timeline. Small synchronization errors immediately become audible, so audio behavior had to be tested as carefully as the interface.

What I Learned

The most important lesson was that useful AI music tooling is not only about generating better audio. It is about preserving creative agency.

Language works best here as a fast way to create musical material—not as a replacement for arrangement, listening, editing, and judgment.

I also learned that local-first AI requires honest product design. Model readiness, downloads, storage, provenance, recovery, and failure states must be visible. “Runs locally” should describe a real, testable boundary rather than remain a marketing phrase.

VibeSeq is my attempt to make AI feel less like a slot machine and more like an instrument:

Compose music with words. Shape every part.

Built With

• electron
• javascript
• python