Rawchunks

Create at your own conditions. Generate and edit your music, and earn with every remix and stream. Or build your own audio product on our open APIs, with no subscription required.

Comment

Rawchunks

Create at your own conditions. Generate, edit, live-code, and license music in an open-source studio, or build your own audio product on the same APIs.

Submitted to the Nebius x NVIDIA Global AI Hackathon. Built from scratch in 6 weeks during the hackathon.

Live Product

Test with 100 Free Credits

You can run full song generations, stem separations, range edits, and live-code agents on app.rawchunks.space without a card or wallet deposit:

  1. Sign in at app.rawchunks.space.
  2. Click SESSION in the top bar and open CAMPAIGN.
  3. Claim 100 free Credits (enough for 20 Fast song generations or dozens of surgical edits).

Why We Built This

Most closed platforms treat music creation like a slot machine: you type a prompt, get a baked stereo file, and if bar 9 has the wrong vocal phrasing or the drop needs a heavier bassline, you throw the track away and roll again. You can't fix four bars in context, isolate or layer stems on a timeline, write MIDI or live-coded synths alongside the audio, reuse a portable voice model you actually own, or build your own app on their stack.

We built Rawchunks in 6 weeks to let both "vibe-creators" and professional musicians/producers have complete control over their entire music production cycle - from generation, through editing, licensing and publication. Proving that an open-source stack can match or beat closed commercial giants on generation speed and audio fidelity while going much further on granular editing and composability. Every model in our pipeline is open-weights, running on our own Kubernetes GPU fleet and exposed through a browser DAW and an open API.

Our endpoints are fully open and with thorough Dev Guides - allowing businesses, startups and indie devs to add music-generation and editing capabilities to apps and agents out of the box, without getting stuck in subscriptions. Paying per-usage in credits, via USDC or credit card, with verifiable x402 calls.

How the Stack Works

Full technical documentation lives in the AI & ML Overview, Model Inventory, GPU Inference Runtime, Queueing & Concurrency, Songwriter Agent, and Algorithmic Composer.

1. Open-Weights Audio Engine on Kubernetes & NVIDIA L4 GPUs (backend/worker)

We run four workload-isolated worker pools on Google Kubernetes Engine (GKE) backed by NVIDIA L4 GPUs, autoscaled via KEDA against durable Valkey job queues.

Out of the box, diffusion transformers are too slow for interactive DAW editing. Our worker overlay compiles the DiT blocks with torch.compile(dynamic=True) and replaces standard normalization and activation layers with custom Triton kernels fusing RMSNorm, SwiGLU, residual additions, and AdaLN modulation. We also compile rotary embeddings and the chunked VAE decoder, persist TorchInductor and Triton caches across restarts, and pre-warm 120s and 240s latent shapes at startup.

During inference, weights stay pinned in VRAM (bfloat16, Flash Attention / SDPA). Because text and lyric conditioning stay fixed across denoising steps, we cache cross-attention K/V projections on step 0 and reuse them across the rest of the trajectory, while restricting Classifier-Free Guidance to the active step window. Each worker also keeps a 2 GiB scratchpad, a 32-file download cache, an 8-entry waveform tensor cache, and a 16-entry CPU VAE latent cache, so consecutive range edits on the same song skip downloading, decoding, and re-encoding altogether.

2. Songwriter & Live-Code Agents on Nebius Token Factory (backend/agents)

Two studio agents run on Nebius Token Factory using nvidia/Nemotron-3-Ultra-550b-a55b and nvidia/Nemotron-3.5-Super-VL with thinking disabled (enable_thinking: false) so tool calls stream back in under a second.

The Multimodal Songwriter (backend/agents/vibe) accepts a text prompt, an image, or a 30-second microphone hum/speech recording. If an image is attached, Nemotron-3.5-Super-VL translates its visual mood into a musical brief while a parallel local DSP pipeline (YIN pitch tracking, Krumhansl-Schmuckler key/scale profiling, onset autocorrelation BPM) and Dual-STT (ink-whisper with non-lexical hum filtering) analyze any recorded audio. Nemotron-3-Ultra then inspects compact studio libraries on demand, streams live UI field updates (set_title, set_prompt, set_vocal_config, set_style_tags, ...), and streams bracket-tagged multilingual lyrics token-by-token while cover art generates in parallel.

The Directed-Graph + ReAct Strudel Composer (backend/agents/strudel) writes and transforms playable Strudel synth, bass, chord, and drum code inside the DAW Editor. Before calling Token Factory, a <1ms local directed graph resolves the song's key, scale, BPM, active section energy (intro, verse, build, drop, breakdown), aligned lyrics, and reference stem role, pre-mounting only the required slices of four music-theory libraries and cutting prompt tokens by ~60%. Nemotron-3-Ultra then runs a 3-turn streaming ReAct loop where set_strudel_pattern validates syntax and Web Audio DSP methods in-loop, feeding any error back to the model for immediate repair before code hits the browser REPL.

3. Browser DAW, Open Studio API & Onchain Licensing (app, backend, blockchain)

Interactive playback, arrangement, and mixing run locally in the browser (React 19, TypeScript, Web Audio API, Tone.js) so timeline work never waits on GPU queues. Master audio, separated stems, and candidate takes share one timeline where selecting any region opens AI operations (Rework, Layer, Extract, Separate, Extend, Restyle, Accompany) alongside a polyphonic Keys piano roll, 16-step Drums sequencer, Strudel live-code REPL, Sampler, 3D Spatial Panner, 8-stage FX Rack, offline WAV bounce, and git-style project History commits.

Every Studio and Editor operation is also exposed through our APIs (REST + SSE), supporting both standard rk_live_* API keys and accountless HTTP 402 (x402) USDC micropayments on Solana and Base. When creators publish tracks or sell remix licenses in the Market, revenue splits and permissions are enforced onchain via our royalties.sol smart contract, which sends 80% to the artist and 20% to the platform.

Repositories

Repository Stack Scope
rawchunks/app React 19, TypeScript, Vite, Web Audio API, Tone.js Web Studio, multitrack DAW Editor, Strudel REPL, Voices, and Market
rawchunks/backend Fastify 5, TypeScript, Drizzle, PostgreSQL, Valkey, PyTorch, Triton, vLLM, KEDA API server, GPU worker overlay (worker/overlay), Nebius Token Factory agents (agents/vibe, agents/strudel), queues, and x402 billing
rawchunks/blockchain Rust (Anchor / Solana), Solidity (Foundry / Base) Onchain song registration, remix licensing, and royalty split contracts
rawchunks/docs Mintlify, MDX User guides, AI/ML deep dives, and OpenAPI reference (docs.rawchunks.space)
rawchunks/landing React 19, TypeScript, Vite Product showcase (rawchunks.space)
rawchunks/submission Markdown Hackathon overview and entry point

Local Quickstart

# 1. Backend API & Agents
git clone https://github.com/rawchunks/backend.git
cd backend && npm install
cp .env.example .env   # Configure NEBIUS_API_KEY, DATABASE_URL, VALKEY_URL, R2 credentials
npm run dev

# 2. Web Studio & DAW
git clone https://github.com/rawchunks/app.git
cd app && npm install
npm run dev

Built With

  • ace-step-1.5-xl
  • cartesia
  • cloudflare-r2
  • demucs
  • elevenlabs
  • fastify
  • keda
  • kubernetes
  • nano-banana
  • nebius-token-factory
  • nemotron-3-ultra
  • nemotron-3.5-super-vl
  • neon
  • node.js
  • nvidia
  • nvidia-l4
  • pytorch
  • solana
  • strudel
  • triton
  • typescript
  • valkey
  • vllm
  • x402
  • zustand
Share this project:

Updates

Submission history