Shaddy

GPU coding unlocks a whole new world, for digital artists and programmers. Yet access to the frontiers are barred behind unintuitive paradigms and lack of resources. Shaddy opens that door for all.

Comment

Inspiration

Scratch - Everyone on the team started their programming journey with scratch and it has allowed us to access a completely new world. Shaddy is our effort to open the next set of doors for all into the world of coding for the GPU.

What it does

Shaddy, built on WebGL2 and React, is a visual shader composer that makes GPU shader art as approachable as snapping puzzle pieces together. You drag cards — gradient, noise, radial mask, palette, vignette, and 140+ more — into a chain. Each snap instantly recompiles a GLSL fragment shader and renders it live on the canvas. The code and the canvas are the same surface: drag a slider on any card and the GLSL updates; edit the raw GLSL and the blocks update back. A Lens Stack view breaks your shader into a column of thumbnails — one per operation — so the invisible GPU math becomes visible. Finished? Hit share, and your entire shader is encoded in a URL no backend required. Works on desktop and phone, same codebase.

How we built it

  • Renderer — raw WebGL2 (no Three.js) running a fullscreen quad. Hot-reloads the fragment shader on every card change, targets 60 fps on mid-range phones with adaptive devicePixelRatio throttling.
  • Cards compiler — a pure-TypeScript pipeline that takes an ordered list of card nodes, resolves their uniform declarations, and emits a single GLSL main() body. Each card is a GLSL snippet with typed input/output pins; the compiler wires outputs to inputs and inlines constants.
  • Bidirectional editor — CodeMirror 6 + @shaderfrog/glsl-parser. The AST is re-parsed on every keystroke; numeric literals get drag widgets; vec3(r,g,b) literals get
  • Lens Stack — re-runs the compiled shader N times, inserting a discard after line K each pass, and captures each FBO frame as a thumbnail strip.
  • Mobile — CSS pointer: coarse + touch events; pinch-to-zoom on the canvas; long-press to scrub numerics; the whole layout reflows into a bottom-sheet drawer on narrow screens.
  • Share — LZ-compressed JSON of the recipe serialised into the URL hash. No server, no accounts.
  • Stack — React 19, TypeScript, Vite, Zustand, Tailwind, CodeMirror 6, react-colorful.
  • Bidirectional binding is a hard parsing problem. Keeping the CodeMirror document, the GLSL AST, and the card state in sync across all three directions (slider → source, source → card, card → canvas) without infinite loops required careful ordering of reactive updates and a stable marker system that survives arbitrary edits.
  • WebGL2 on mobile. Fragment shaders that look fine on desktop can drop to 15 fps on a mid-range phone. We had to add a DPR throttle, a frame-budget monitor, and a compile-error recovery path that degrades gracefully rather than freezing the canvas.
  • Lens Stack FBO budget. Rendering N offscreen passes per frame for a 10-card chain was initially slow enough to block the main frame. We moved the stack renders to an idle-callback queue and only regenerate the thumbnails when the recipe actually changes.
  • Deferred ambition. The original spec included a photo→shader gradient-descent feature (FastAPI + PyTorch backend, differentiable shader templates, WebSocket streaming). We cut it mid-build when we realised it would eat the hackathon timeline — pivoting to a fully frontend-only product was the right call, but re-scoping under time pressure was stressful.

Challenges we ran into

  • Bidirectional binding is a hard parsing problem. Keeping the CodeMirror document, the GLSL AST, and the card state in sync across all three directions (slider → source, source → card, card → canvas) without infinite loops required careful ordering of reactive updates and a stable marker system that survives arbitrary edits.
  • WebGL2 on mobile. Fragment shaders that look fine on desktop can drop to 15 fps on a mid-range phone. We had to add a DPR throttle, a frame-budget monitor, and a compile-error recovery path that degrades gracefully rather than freezing the canvas.
  • Lens Stack FBO budget. Rendering N offscreen passes per frame for a 10-card chain was initially slow enough to block the main frame. We moved the stack renders to an idle-callback queue and only regenerate the thumbnails when the recipe actually changes.
  • Deferred ambition. The original spec included a photo→shader gradient-descent feature (FastAPI + PyTorch backend, differentiable shader templates, WebSocket streaming). We cut it mid-build when we realised it would eat the hackathon timeline — pivoting to a fully frontend-only product was the right call, but re-scoping under time pressure was stressful.

Accomplishments that we're proud of

  • The bidirectional binding actually works. Drag a slider → GLSL updates → canvas redraws. Type directly in the GLSL → cards update → canvas redraws. The loop is lossless; no state is thrown away.
  • 140+ composable cards covering gradients, noise, color palettes, SDF shapes, blending modes, domain warping, 3D raymarching, reaction-diffusion, and more — all live-compilable in the browser.
  • Lens Stack as a teaching tool. Clicking "show steps" turns abstract GPU math into a visible column of intermediate images. It's the most common "whoa" moment in user testing.
  • Zero-backend share. Full shader state round-trips through a URL. Open it on your phone, the shader is already running.
  • Responsive with real mobile gestures — the same codebase works on a phone with touch scrubbing, pinch-to-zoom, and a drawer-based layout with no separate mobile build.
  • A gallery of curated recipes — each tagged with a real-world use case ("sci-fi loading screen", "game menu background") to show that shaders are practical, not just academic.

What we learned

  • The canvas-code binding is the product. Everything else — gallery, templates, export — is packaging. The moment users realise that dragging a slider rewrites the GLSL, the mental model clicks. We underestimated how much design work it would take to make that moment obvious.
  • Mobile WebGL is a first-class constraint, not an afterthought. Designing the DPR throttle and frame-budget monitor early saved us significant late-stage debugging. Future GPU-heavy web apps should budget a dedicated mobile performance pass from day one.
  • Parsing GLSL in the browser is tractable but fragile. @shaderfrog/glsl-parser covers the common cases well, but edge cases in the AST (implicit conversions, preprocessor directives, nested swizzles) require defensive error handling throughout the binding layer.
  • Scope cuts are product decisions. Dropping the PyTorch backend felt like a failure at first. In hindsight it produced a tighter, faster, more reliable demo — and a clearer product identity: "everything runs in your browser, no sign-up, no server."

What's next for Shaddy

  • CPU vs GPU live benchmark — show a side-by-side ms/frame comparison of the same recipe running on CPU (TypeScript) vs GPU (GLSL) to make the "200× faster" claim visceral and defensible.
  • Export to game engines — a dropdown that emits the compiled GLSL as a Unity .hlsl shader, a Godot 4 visual shader, or a ready-to-paste Three.js ShaderMaterial. The compiler already emits clean GLSL; the translation layer is the remaining work.
  • Photo → shader gradient descent — the deferred feature: upload an image, a differentiable version of your recipe's parameters is optimised via PyTorch on a serverless GPU (Modal/Replicate), and the result streams back as a parameterised, infinitely-zoomable shader you can keep editing.
  • "Recreate this" tutorial overlays — a 4-step guided overlay on gallery recipes that walks a complete beginner through building that specific shader from scratch, bridging the gallery and the /learn lesson path.
  • Audio reactivity — a u_audio uniform sourced from the mic FFT so shaders can pulse to music in real time.

Built With

Share this project:

Updates