Last Stand

A wave of cursed spirits descends on the sacred garden. Stand at its heart and resist.

Comment 1

Yes I know the IMU isn't a microcontroller I was talking about the ESP (I was ad libbing leave me alone).

Inspiration

We wanted to feel like a Jujutsu Kaisen sorcerer, not press a button labeled "attack," but actually throw a punch and watch a cursed spirit launch across the screen, or make a flat blade-hand gesture and cleave an enemy clean in two. Most motion-control demos feel like crap. We wanted real physics, real impact, and a signature moment people would replay. With Rork as our sponsor, we set out to build a combat game that blends computer vision, custom hardware, and physics into something that feels genuinely powerful.

What it does

Last Stand is a browser-based 3D combat game set in a cel-shaded Japanese garden, where you defend against a horde of approaching curses. It splits control across three independent channels so it's always playable:

  • Aim with the keyboard or by turning your head (webcam head-tracking).
  • Gestures via your webcam: a flat blade hand cleaves an enemy (a clean, physics-based bisection at a random angle), an open palm shoves back a group.
  • Punch with a real ESP32 + IMU forearm band: a physical thrust is detected over WiFi and launches the targeted curse, with knockback scaled to how hard you hit.

Land kills to charge Domain Expansion (triggered by a hand sign you record yourself), and chain a big one to unlock Furnace, a giant pillar of flame. There's also a 1-on-1 boss duel mode where you circle a Cursed Shogun and must time parries and manage a guard stamina bar. A Rork-built companion web app wraps the whole thing with menus, a gesture guide, band setup, and a leaderboard.

How we built it

We built Last Stand as two pieces that come together into one experience: a Rork-powered companion app as the hub, and a real-time 3D combat engine it launches.

  • Rork (our front-end and hub): We used Rork's AI app builder to generate the entire companion web app, a polished React + Tailwind front-end with a home screen, an interactive how-to-play and gesture guide, a hardware band-setup flow, and a persistent leaderboard. Rork scaffolded the whole multi-screen app, navigation, and UI component system from prompts, and it's the shell players actually open: the combat game itself is embedded right inside Rork's Play screen, so the game, menus, gesture guide, and scoreboard all live in one Rork-built product. Rork went from idea to a working, themed app in minutes, which let us spend our time on the hard real-time and hardware problems.
  • The combat engine (embedded in the Rork app): Inside that shell runs our browser game : Three.js for rendering, Rapier (WASM) for physics, and MediaPipe Hand + Face Landmarkers for gesture and head-tracking control. The signature clean cleave uses per-frame clipping planes for arbitrary-angle bisection, with cel/toon shading, ink outlines, and bloom for the anime look.
  • The hardware: An ESP32 + IMU forearm band, with firmware flashed via arduino-cli, running a WebSocket server on a static IP over a phone hotspot. It detects punches as accelerometer spikes and streams them to the game, fully wireless on a USB battery.

Rork tied it all together into a single, demo-ready front door, while the embedded engine and band delivered the gesture- and motion-controlled combat behind it.

Challenges we ran into

  • The hardware fought back first. A charge-only USB cable powered the board but never enumerated
  • Our "MPU9250" was secretly an MPU6500, no magnetometer, which killed our directional-thrust stretch goal. The Arduino library refused the chip entirely, so we threw it out and talked to the IMU over raw I2C registers, which works across the whole sensor family.
  • The clean cut got messy. Our stencil-based "cut cap" needs a watertight mesh, but our enemy bodies were merged from overlapping parts so the cut face bled through as huge squares. We switched to double-sided clipped solids.
  • A 180° bug had the player facing away from the boss in the duel (the aim arrow pointed backwards).
  • Browser security vs. hardware: browsers block ws:// from HTTPS pages, so the band only works on http://localhost. We designed every input channel to be independent so the game is fully demoable even if the camera or band fails.

Accomplishments that we're proud of

  • A juicy, complete core loop that's fun with zero hardware and levels up when you strap on the band — no single input is a hard dependency.
  • The signature clean cleave: real arbitrary-angle bisection with both halves tumbling under physics.
  • A forearm band that actually works wirelessly off a battery, with sub-200ms punch-to-impact latency after tuning.
  • A full anime visual transformation: Japanese garden, samurai player, glowing curses, sakura, bloom.
  • Two ultimates (including a player-recordable Domain sign) and a complete boss duel with parry + guard, all built on the same gesture/IMU inputs.

What we learned

  • Hardware lies and the real world is hostile — cheap IMUs misreport their part number, cables silently fail, power banks auto-shut-off, and DHCP hands out new IPs. We learned to verify (WHO_AM_I, I2C scans) and to harden (static IP, auto-reconnect) instead of trusting.
  • Browser security model matters — secure contexts, getUserMedia permissions, and HTTPS/ws:// mixed-content rules shaped our whole deployment.
  • Real-time 3D "juice" techniques — clipping-plane slicing, stencil caps, toon + outline NPR, hit-stop, slow-mo, screen shake, and impact frames are what make hits feel good.
  • Design for resilience — at a live demo, the win is a system where any one part can fail and the experience still lands.

What's next for Last Stand

  • Directional thrust — swap in a true 9-axis IMU (with magnetometer) so a physical left-punch sends the left enemy flying, using sensor-fusion orientation.
  • More bosses, combos, and difficulty tiers, plus a co-op or versus mode.
  • More shock value — mapping the player's webcam face onto the curses, a slow-mo killcam, and a voice-activated finisher.
  • Deeper Rork companion — live leaderboard sync from the game and a richer 3D menu.

Built With

  • arduino
  • c++
  • css
  • dusk-sky
  • esp32
  • getusermedia
  • git
  • github-categorized-breakdown-languages-javascript-?-the-game-(three.js/rapier/mediapipe)-typescript-?-the-rork-companion-app-c-/-c++-?-esp32-firmware-(band.ino)-glsl-?-custom-shaders-(fire-pillar
  • glsl
  • html
  • i2c
  • javascript
  • localstorage
  • mediapipe
  • mpu6500
  • python
  • radix-ui
  • rapier
  • react
  • react-router
  • rork
  • tailwindcss
  • three.js
  • toon-ramp)-python-?-a-small-serial-reading-helper-for-imu-debugging-html-/-css-frameworks-&-libraries-three.js-?-3d-rendering-rapier-(@dimforge/rapier3d-compat)-?-wasm-physics-mediapipe-tasks-vision-?-hand-+-face-landmarker-(gestures
  • typescript
  • vite
  • webassembly
  • webgl
  • websockets
+ 17 more
Share this project:

Updates