Brawlmegle

Inspiration

The developers of this project grew up on Wii Sports boxing and gesturing in front of Xbox Kinect. Next came VR boxing but to us, it felt like it was missing something. With Brawlmegle, we mixed old with new, added the omph which VR Boxing lacks, the effect and impact that classic fighting games, Street Fighter and Tekken, gives.

What it does

Brawlmegle is a webcam fighting game where you fight as yourself:

Get scanned: stand in front of your laptop camera, and in about 40 seconds our own GPU server turns you into a stylized, rigged 3D fighter. You can even watch it form live.
Throw hands: punch, hook, uppercut, block, duck, dodge, kick. The camera reads what you meant to do, and the fight engine decides what actually happens in order to create an awesome strategic and visual experience.
And of course it has the flashy arcade stuff we all love like hero entrances, VS screens, and even live AI commentary powered by ElevenLabs.

How we built it

The game is React, TypeScript, and Three.js in the browser, so there's nothing to install. Body tracking happens via Google's MediaPipe and runs in a Web Worker at around 30 frames per second.

Our secret sauce is the intent reader. Instead of mapping your skeleton onto the model, it watches your motion and decides "that was a right hook".

The game is online peer-to-peer via WebRTC, with a small matchmaking server and relay hosted on a tiny AWS EC2 instance. The fight engine runs on one of the laptops: it acts as the referee, and the other one just mirrors it and collects input from the second player.

For the fighters, we built our own "forge":

GPT writes your fighter's backstory and draws a character sheet from your scan.
TRELLIS.2, an open-source 3D model we run on our own AWS GPU, sculpts it into a 3D fighter in seconds.
Our own Blender auto-rigger gives it a skeleton so it can actually fight.
A last step paints your face back on, so it looks a little more recognizable.

We were also fortunate to have a talented Blender wizard on our team, who built an incredible Mount Rainier arena so you're fighting right here in the PNW.

Challenges we ran into

The hardest challenge was the intent reader. We kept having issues with fake moves, wrong moves, and even huge delays, which defeated the dream of building something punchy. But as we kept recording fights and running tests, we were able to tune a config system that we believe got to a point where the gameplay is actually fun.

The other big one was the 3D fighters. Our 3D generation provider ran out of credits mid-hackathon, so we spun up our own GPU server, which ended up faster. Then the models came out hollow, so dresses and hoodies broke the rigging, and we had to write our own rig checker and fixes until every fighter worked.

Accomplishments that we're proud of

Witnessing going from webcam scan to a playable 3D version of yourself in under a minute never gets old.
Real fighting-game feel from just a camera: combos, counters, guard breaks
Omegle-style matchmaking with live cameras, fully peer-to-peer
An AI live commentator that's actually aware of what's going on in the game.