Strudel Your Code

Hack & Roll 2026 | Team Better Call SQauL

Turn your coding sessions into live generative music performances

What is this?

A dual-component system that transforms your live coding activity into generative audio:

VS Code Extension - Tracks 50+ real-time coding metrics (keystrokes, commands, idle time, debugging sessions, etc.) and uses OpenAI to generate Strudel music patterns based on your coding intensity
Visualizer - A Tauri desktop app featuring a 3D dance floor with animated agents, musical instruments, and live Strudel audio playback. Use Performance mode to see and hear your code. Use Sandbox mode to record your own audio.

How They Work Together

 VS Code Extension                         Visualizer (Tauri)
 +-----------------------+                 +-------------------------+
 |                       |                 |                         |
 |  MetricsTracker       |                 |  Three.js Scene         |
 |  - Keystrokes         |   Strudel       |  - Animated agents      |
 |  - Commands           |   Patterns      |  - Piano, guitar, amp   |
 |  - Idle/focus time    |  ----------->   |  - Dance floor          |
 |  - Debug sessions     |   (Manual or    |                         |
 |                       |    file-based)  |  Strudel Audio Engine   |
 |  OpenAI Integration   |                 |  - Live pattern eval    |
 |  - gpt-5-mini         |                 |  - Preset instruments   |
 |  - Generates Strudel  |                 |  - Session recording    |
 |                       |                 |                         |
 +-----------------------+                 +-------------------------+

The extension monitors your coding activity and generates music that reflects your workflow:

High intensity coding (rapid typing, many commands) → faster, energetic patterns
Reading/reviewing code → ambient, slower patterns
Taking breaks → calm, minimal notes
Debugging sessions → influences pattern complexity

The visualizer provides an immersive 3D environment where chibi-style agents dance on a colorful checkered floor alongside instruments (piano, guitars, amplifier), with live Strudel audio playback.

What is Strudel?

Strudel is an open-source live coding environment for making music in the browser. Inspired by TidalCycles, it lets you create complex, evolving musical patterns by writing JavaScript code.

Why Strudel?

Browser-native — No installation required; runs entirely in the Web Audio API
Pattern-based — Music is described as patterns that cycle over time
Composable — Small functions chain together to build complex arrangements
Live-codable — Change the code, hear the results instantly

Key Concepts

Concept	Description	Example
Patterns	Sequences of events over a cycle	`note("c4 e4 g4")`
Sounds	Oscillators and synths	`sound("sawtooth")`, `sound("triangle")`
Mini-notation	Shorthand for rhythms	`"c4 [e4 g4]"` = c4 for half, e4 & g4 split the other half
Transformations	Modify patterns in time	`.slow(2)`, `.fast(1.5)`, `.rev()`
Effects	Audio processing	`.gain(0.8)`, `.pan(0.3)`, `.lpf(800)`
Stacking	Layer patterns together	`stack(melody, bass, drums)`
Euclidean rhythms	Algorithmic beat distribution	`.euclid(3, 8)` = 3 hits spread over 8 steps

Example Pattern

// Layered ambient pattern with stereo panning
stack(
  note("c3 eb3 g3").sound("sawtooth").slow(2).gain(0.6),
  note("c4 g4").sound("triangle").gain(0.4).pan(0.3)
)

codeEssembler generates patterns like this automatically — the AI interprets your coding metrics and translates them into musical intensity, rhythm, and texture.

Features

VS Code Extension

Real-time tracking of 50+ coding metrics
AI-powered Strudel code generation via OpenAI
Sidebar UI with:
- Audio controls and live code editor
- Adjustable evaluation cadence (10-300 seconds)
- Music preferences (genre, mood, sound palette)
- Live metrics dashboard

Visualizer

Three.js rendered dance floor with animated agents
Multiple instruments (piano, acoustic/electric guitar, amplifier)
Strudel audio engine with preset patterns
Agent selection and choreography system
Session recording with WAV/MP3 export
Agent chat bubbles with AI-generated quips

Prerequisites

Node.js v22.13.1+
VS Code 1.108.1+
Rust (for Tauri builds)
OpenAI API Key

Quick Start

VS Code Extension

cd vscode-ext
npm install

# Create .env file
echo "OPENAI_API_KEY=your-api-key-here" > .env

npm run compile

Press F5 in VS Code to launch the extension in debug mode.

Visualizer

cd visualizer
npm install

# Create .env file  
echo "VITE_OPENAI_API_KEY=your-api-key-here" > .env

# Development (Vite only)
npm run dev

# Full Tauri desktop app
npm run tauri dev

Project Structure

hack-n-roll-2026/
├── vscode-ext/              # VS Code extension (TypeScript)
│   └── src/
│       ├── extension.ts     # MetricsTracker + activation
│       ├── webview.ts       # Sidebar UI
│       └── strudel/         # OpenAI integration
│
├── visualizer/              # Tauri desktop app
│   ├── src/
│   │   ├── main.js          # Three.js 3D scene
│   │   ├── audio.js         # Strudel playback
│   │   └── sessionRecorder.js
│   └── src-tauri/           # Rust backend (minimal)
│
└── strudel_ref.md           # Strudel API reference

Tech Stack

Component	Technologies
VS Code Extension	TypeScript, ESLint, esbuild, OpenAI SDK
Visualizer Frontend	Vanilla JavaScript, Three.js, Vite
Visualizer Backend	Rust, Tauri 2.x
Audio Engine	Strudel
AI	OpenAI gpt-5-mini