HackStash

An elaborate, immersive Heist simulator.

Comment

Hack Stash is a getaway game set in downtown San Antonio. You pull off three heists at real landmarks, and each one makes you do something in real life: fool a face scanner, crack a physical vault on the table, and talk an AI bank agent into giving you a code over the phone.

You drive through a 3D downtown rebuilt from real OpenStreetMap data, with a police cruiser chasing you the whole time. You can steer with the keyboard, or hold your hands up to the webcam and turn them like a steering wheel.

Why it's a heist

Every heist movie has the same crew: someone who cons the guard, a lookout, a safecracker, a smooth talker on the phone, and a wheelman. Hack Stash makes you play every one of those roles for real, not by pressing a button.

  • The con: you fool a real face scanner at the River Walk.
  • The lookout and the safecracker: at the Alamo, one person blinds a real security camera while the other cracks a real vault on the table.
  • The social engineer: at the Tower, you call a real phone number and talk an AI bank agent into sending you a code.
  • The wheelman: you drive the getaway through a real downtown San Antonio with the police right behind you.

The vault doesn't open in the game until it opens on the table.

The three heists

1. The River Walk: "Face the Vault"

A guard at the River Walk checkpoint scans your face through the webcam. All you get is a riddle:

"Acknowledge the guard, signal twice, and look pleased."

The checklist stays hidden until you pass each step, so you have to work out the answer: nod, blink twice, smile, in that order. Your webcam streams at 30 fps to a face-analysis server running Presage SmartSpectra, which picks up each gesture as you make it. Get it wrong three times and the alarm goes off and your wanted level climbs. Get it right and you walk away with $5,000.

2. The Alamo: "Crack the Vault"

This vault is a real box on the table: an Arduino with a light sensor, a joystick, a 16x2 LCD and a servo holding the door shut.

  1. Blind the camera. The light sensor is the security camera. Cover it with your hand and keep it covered, or the LCD says SPOTTED!.
  2. Repeat the pattern. The LCD flashes a random four-move unlock sequence. Repeat it on the joystick. One wrong move and it plays again.
  3. Listen. Once the pattern is cracked, the vault says a secret keyword out loud through a Bluetooth speaker.
  4. Type it in. Your partner at the laptop types the keyword into the game, and the servo swings the vault door open.

It takes two people: one on the vault, one on the keyboard.

3. Tower of the Americas: "The Callback"

The bearer bonds are in the Tower, behind a bank-grade lock. The only way in is through the bank's fraud line.

You call a real phone number and get Margaret, an AI voice agent built on ElevenLabs and connected through Twilio. She asks identity questions, and you have to convince her you're the account holder (or find the staff override phrase). If she believes you, she sends a one-time code "to the device on file." That device is the vault's LCD, and the code is only on screen for 8 seconds. Read it, then enter it on the joystick within 30 seconds. Up and down change the digit, left and right move between positions.

Enter it correctly and Margaret approves the transfer, the vault opens, and you get $12,000. If you sound fake or burn three codes, she gets suspicious, calls the cops, and your wanted level jumps.

The getaway

  • Steer with your hands. MediaPipe tracks both hands in the browser. Turn them like a steering wheel to steer, curl your fingers to accelerate, and open both hands to reverse.
  • A real San Antonio. Every building, road, the river, the trees and the three landmarks are generated in code from OpenStreetMap data. There are no hand-modeled city assets.
  • Police that drive like you do. The police cruiser uses the same A* road graph as your GPS route and recalculates its path every 0.45 to 0.75 seconds. It can't cut through buildings any more than you can. If it touches your car, you're caught.
  • Noir mood. It's always night and always raining, with a chase soundtrack, a minimap and a glowing route on the road leading to your next target.

How it works

Hack Stash architecture

Hack Stash is built from five parts. Everything connects through a single Python game server on Render, which holds the loot, the wanted level and the state of each heist. The browser game talks to it over Socket.IO. For the River Walk, the browser also streams webcam frames to the Presage face scanner on our laptop. For the Tower, Twilio and ElevenLabs handle the phone call, and Margaret reaches the server through webhooks. The vault rig (Raspberry Pi + Arduino) connects to the same server, so a code made on the server can show up on a physical LCD, and a move on the joystick can open the vault in the game. The arrow colors in the diagram show which parts each heist uses: green is the River Walk, orange is the Alamo, purple is the Tower.

Game: React 19, React Three Fiber and Three.js, built with Vite and deployed on Vercel. We pulled downtown San Antonio from the Overpass API and turned it into procedural geometry. The car physics, collision and the A* road graph are all our own code. Hand tracking uses MediaPipe Hand Landmarker, smoothed with a One Euro filter so the steering doesn't jitter.

Game server: Python, Flask and Flask-SocketIO on Render, running under Gunicorn. It's the one place that knows the game state (loot, wanted level, heist progress). The Alamo and Tower heists are plain state machines with unit tests and no network code inside them, so we could test every edge case without the hardware plugged in.

Face scanner: a Node.js server running the Presage SmartSpectra SDK. It decodes JPEG frames with sharp and turns Presage's face landmarks and expression scores into nod, blink and smile events. MediaPipe Face Landmarker and OpenCV are a fallback on the Python server.

Vault rig: an Arduino Uno (C++) connected over USB serial to a Raspberry Pi. The Pi runs a small Python relay as a systemd service. It sends sensor readings up to the game server over Socket.IO, sends LCD and servo commands back down to the Arduino, and speaks keywords with espeak-ng. The Pi never decides anything on its own, so we could change the heists without reflashing anything.

Margaret: an ElevenLabs conversational agent on a Twilio number. She has five webhook tools on our server: sendOtp, checkOtp, raiseSuspicion, approveTransfer and triggerAlarm.

The design decision we're proudest of

Margaret never knows the code. When she calls sendOtp, the server makes up the code and sends it only to the physical LCD. Her tool just answers "code sent." approveTransfer returns HTTP 403 until the code has actually been entered on the joystick. Margaret decides whether you're believable, but she can't approve the transfer by herself, so there's nothing to trick out of her. You still have to be at the vault.

Challenges we ran into

  • Presage needed more than our server had. The SDK needs Vulkan and a steady stream of about 30 frames per second, and after any gap longer than 2 seconds it rejects every frame that follows. We drop frames when the CPU falls behind and squeeze long pauses down to a single frame step. The free cloud tier still couldn't keep up, so the scanner runs on a laptop and the game reaches it through a Cloudflare Tunnel.
  • Smiles were hard to detect. In our testing, Presage's expression model labeled a neutral face "surprise" about 94% of the time and rarely gave a confident "happy." We added a second signal from the face landmarks: if your mouth gets 10% wider than its resting width, that counts as a smile too.
  • Real-time connections on Render. Gunicorn removed its eventlet worker during the hackathon, and WebSocket upgrades from the Pi kept failing behind Render's proxy. We switched to Gunicorn's threaded worker and had the Pi connect with HTTP long-polling.
  • Hardware problems. We fixed joystick axes that came in swapped, an LCD that flickered when it got too many updates (now limited to 4 per second), and a servo that didn't turn far enough to open the door. We also made "camera covered" stay on for the rest of the sequence, so you don't fail just because your hand slips while you're on the joystick.
  • Making the chase fair. The landmarks are in pedestrian areas, so the route goes to the nearest road access point and never points you through a building. The police get a 4-second grace period and start about 185 m behind you, so you have a chance.

What's next

  • More heists at more San Antonio landmarks, each with its own piece of hardware.
  • Face scanning in the cloud, on a host with enough CPU that the River Walk doesn't need a laptop.

Try it

You can drive and outrun the police right in the browser. The Alamo and Tower heists need our vault rig, and the River Walk needs our face-scan server running, so come try those at our table or watch them in the demo video.

Built With

+ 1 more
Share this project:

Updates

Submission history