-
-
IRL
-
In-game
Inspiration
Boxing is a two-person sport. You can shadowbox alone all day, but the real learning happens when someone is across from you, and finding that someone is the hard part.
That’s the problem that got us. One of us (hi, it’s Anthony) wanted to keep learning Muay Thai but kept missing the gym around school, family, and everything else. No regular partner, no coach, no feedback. Training alone gets lonely fast.
So we built a place to show up. Spacetime Dojo is a browser-based boxing dojo where your body is the controller and there’s always someone to spar. The look and feel is Wii Boxing nostalgia, and underneath it’s a real-time multiplayer game. Your webcam or your AirPods become the gloves. Step into a ring, fight a friend, a stranger, or a bot, or just grab a spot on the side and watch.
What it does
A living dojo. Four rings run at the same time, and you can see them all live. Jump into a ring or spectate, the way you’d wander around a busy gym. Real people to train with. Fight another player in real time through matchmaking, or train with friends from your friends list showing who's online. Your body is the controller. Throw straights and hooks, dodge, and block with your webcam (pose and hand tracking), your AirPods (motion sensors), or the keyboard. The keyboard started as our testing mode and honestly stayed fun. Different bodies, rooms, and gear, no special hardware. Snappy combat. Moves are detected and classified locally, and the model can recognize a punch before it’s finished, which hides network delay. A sparring bot that adapts (early prototype). Instead of picking easy, medium, or hard, the bot is meant to watch how you fight and adjust. It’s the roughest part of what we built, and we ran out of time to make the adaptation visible. It’s where we want to take it next.
How we built it
We built everything around one idea: a punch should feel instant, even when you’re fighting someone across the internet.
SpacetimeDB is our database and our game server in one. Match state, matchmaking, combat resolution, damage, KOs, and player profiles all live in a single TypeScript module. When you throw a punch, the client sends a tiny motion event to a server-side reducer. The server checks stamina and attack timing, then decides whether the punch lands, gets blocked, or misses based on your opponent’s defense. Clients subscribe to that live state and follow what the server says happened. The server is always the source of truth, which keeps fights fair and makes sure spectators see the same fight the fighters do.
The frontend is Next.js and Three.js on Bun, deployed on Vercel. Blender (and Anthony Pham) brought the boxers and the arena to life.
Webcam tracking uses MediaPipe in the browser, split across two Web Workers so the game stays responsive. Your camera feed never leaves your device.
The punch detector is a 1D CNN + GRU we trained in PyTorch, then ported to TypeScript. We tested the two versions frame by frame so the browser behaves like the model we trained. It gets from motion to the server in ≤30 ms.
AirPods give us accelerometer and gyroscope data through Apple’s Core Motion framework. We map it to straights and hooks, and we got a little hacky to track two AirPods at once. We utilized an iPhone as a second server to host the second airpod (as only one airpod can be configured at a time), which we made a whole custom helper applications to run that.
Challenges we ran into
Latency. In a fighting game, a few extra milliseconds ruin the feel. We spent a lot of time tuning detection thresholds, trimming classification delay, and sending actions to the server the moment they were detected instead of batching them.
Scope creep. We started out building an adaptive AI sparring bot. Then multiplayer and the dojo took over, and honestly that turned out to be the better idea. At one point we were juggling multiple CV pipelines, a standalone arena, and the adaptive bot, so we re-scoped around a core loop that actually feels good.
AirPods calibration. Turning raw motion into reliable punches was harder than we expected. A change in grip or orientation changes how the same punch looks, and a wrist flick or the return to guard can look like the start of another punch. We added a calibration step when your AirPods connect, plus movement vectors, to tell real punches apart.
Accomplishments that we’re proud of
It feels like boxing. After lots of tuning to tell hooks from straights, we landed our first clean left hook using only an earbud, and the whole table cheered. Two AirPods, tracked at once. It took some creative hacking, and it works. An arena that feels alive. Four simultaneous rings, bot exhibitions, and live spectating make the dojo feel like walking into a gym. Open a link and start throwing punches. The webcam pipeline runs locally in the browser, with no video uploads and no install.
What we learned
Real-time computer vision is harder than it looks. Lighting, positioning, and movement change everything, and a lot of the work is tuning until it feels intuitive. Latency is a feature. We got deliberate about what runs locally, what goes over the network, and what the server needs to know. A prototype isn’t a product. Getting the detector working in Python was only the start, and porting it to TypeScript taught us to test across implementations, since one frame can change the outcome of a fight. Server authority and a responsive feel aren’t opposites. Fast local detection plus a server with the final say gave us a game that feels instant and stays fair. Scope matters as much as execution. We cut a lot, and getting one mechanic to feel right beat having ten half-finished ones.
What’s next for Spacetime Dojo
Make the dojo social. Ranked matchmaking, fight history, and progression, plus ways to find and stick with the people you train with. A real coaching mode. Live form feedback like “you’re dropping your right hand after the jab,” then drills that build from fundamentals to combos. An adaptive bot you can actually see. We want an eight-spoke wheel over its head that turns every time it adapts to you. It was on our board and lost the priority fight. More moves. Uppercuts, body shots, rolls, and parries. Better tracking everywhere. Low-end laptops, phones, and calibration that handles bad lighting and cramped rooms.
Log in or sign up for Devpost to join the conversation.