Boxing Canvas

Inspiration

Boxing and painting are more similar than you might think. A punch has direction, speed, weight, and timing. A brushstroke has the same qualities. That overlap became the starting point for Boxing Canvas: a boxing match where the fight produces the artwork.

I was also interested in a question from the hackathon theme: what kind of art needs technology to exist? A finished digital painting would not be enough. I wanted the code to be a part of the performance. The player, the opponent, the physics, the camera, and the sound system all add to a piece that is generated during the match and cannot be reproduced ever again.

What the project does

The player fights El Chupacabra inside a 3D boxing ring. Jabs, hooks, stretch punches, parries, dashes, and projectiles each create different marks. Their position, color, size, direction, and energy come from the combat event that produced them. The arena starts as a fighting space and gradually becomes a moving canvas.

The same actions also make music. Successful hits climb through a C Lydian Dominant scale:

C, D, E, F#, G, A, B$\flat$, C

The combo can continue for 20 hits, with a six-second window between successful attacks. Jabs use short sampled mallet strikes. Hooks add a lower in-scale tone and a physical body thump. Stretch punches sweep upward into the current combo note. Parries, paint shots, and damage each have separate sound materials. Later hits use harder xylophone recordings.

How I built it

The project is a browser-based React and TypeScript application built with Vite. Three.js renders the ring, characters, lighting, camera, and imported GLB animations. A combat state system coordinates movement, attack timing, collision windows, health, opponent behavior, visual impact effects, and the final artwork.

The paint system turns game data into visual parameters. An attack supplies an impact point, gesture type, strength, and color. Those values control the emitted shapes, particles, trails, splashes, and camera response. This keeps the artwork tied to what happened in the fight instead of placing random decoration over the screen.

Tone.js handles the auditory layer. The first version used oscillators and several synths, but the combo melody sounded too electronic, so I replaced it with two velocity layers of xylophone recordings loaded through Tone.Sampler. MembraneSynth, FMSynth, MetalSynth, and filtered noise still provide the body thumps, charged sweeps, parry shimmer, and impact cracks. Compression and a limiter keep stacked sounds under control.

Challenges

The hardest design problem was keeping combat, painting, and music connected. If the effects were too random, the project felt like a boxing game with an art component attached. If every punch produced the same mark and note, it didn't feel like "art". I ended up giving each gesture its own visual and sonic grammar while keeping one shared combo scale.

Audio took several rounds of revision. The original combo began too low, climbed too quietly, and used a fat sine oscillator that made the whole sequence sound like an arcade. Moving the notes into a brighter tone helped, but recorded mallet samples made the larger difference. I then capped the gain curve, used soft and hard sample layers, and let later combo stages feel stronger without making the output unsafe.

Performance was also a key issue in production. The scene combines animated 3D models, particles, paint effects, UI, and real-time audio, so I reused audio nodes, limited effect lifetimes, kept the sample set small, and used sparse note mapping so Tone.js could pitch nearby recordings instead of loading an entire instrument library.

What I learned

Building the front-end UI taught me how hard it is to make a screen look good while it sits on top of a moving 3D game. The title, health bars, controls, combo meter, start screen, and sound wordmark all had to remain readable without covering the ring or distracting from the paint effects. I spent more time than expected adjusting responsive sizes, layering, contrast, typography, pointer behavior, and safe areas. Front-end work is more about how components work with each other rather than by themselves.

I also learned that generative art needs rules the audience can sense. The player does not need to know the MIDI values or particle parameters, but they should notice that a hook paints and sounds different from a jab. Once those relationships became consistent, the fight started to feel like an instrument instead of a game with effects layered on top.