The Appraisal Job

Perfect Heist Kit: Stealth Rover, Wearable Display & Spy Hat

Comment

The Appraisal Job

A walkthrough is only useful if the team can reopen it later. The Appraisal Job is a three-device field kit for an authorized room review. A hat-mounted Raspberry Pi, an ESP32 wrist display, and a Pi rover write the same case ledger. Finish the walk and the hub freezes that ledger into a Defender Report.

Code:

What it does

The hub camera runs at 1280×720, about 12 frames a second. It does not call a model on every frame. A look starts only when the mean absolute difference on a (160 \times 120) gray frame clears the gate for two samples in a row:

$$ \bar{d} > 8 \quad \text{and} \quad t - t_{\text{last}} \ge 2.5\,\text{s} $$

Each look names up to 8 distinct priced objects.

Every exhibit stores a name, a dollar estimate, a source label, a still, and an origin, hat or rover. The dashboard, the dispatch desk, and the wrist all read that one file. The wrist long-polls GET /wrist.json and shows the case number, the running total, and the top five by value_usd. Names on the small screen are cut to 12 characters.

The rover can scout on its own. It queues up to 50 finds and posts them with POST /api/exhibit. Drive commands are a different route, capped at 3 seconds, and an idle rover dead-reckons home. A printed ArUco tag (DICT_4X4_50, id 0) is the home marker, because wheel slip makes pure dead reckoning drift.

Reveal is a button on GPIO17, or a click on the desk. The report copies every exhibit and one crop into reports/<id>/ before the live case resets. If that write fails, the case stays. The dollar total is estimated asset value. It is not a risk score, and it is not a prediction of loss.

How a price gets attached

Sold comps are off unless SerpAPI is turned on. When it is on, the hub pulls eBay sold listings, keeps titles that share the model tokens, and takes the median. A generic word like "headphones" is not enough to price an item.

Step When it runs What gets stored
eBay sold comps SerpAPI enabled, titles share model tokens median price, estimated = false
Model quote no comp used street price from gpt-4.1-mini, not MSRP
Vision number the quote also fails the look's own number, labeled vision estimate only

Badges skip that ladder. The value stays an access number (price_source = credential), and the two sides of a campus card merge on the printed id instead of becoming two exhibits. Payment cards are refused. A Visa, Mastercard, Amex, or Discover number that passes the Luhn check is masked to the last four before it is stored. Campus and library numbers that fail that check are kept.

Repeated sightings collapse when the name match is above (85), or above (55) inside the same category. Exits dedupe on the full name, so a back door and a side door stay separate. The ledger caps at 500 exhibits.

If no provider key is configured, or offline mode is on, a labeled fixture catalog fills the demo. Those finds are marked fixtures on the report. If the live providers are configured and all of them fail, the hub files nothing. It does not quietly substitute demo items.

Planning is a simulation

Mastermind is a second mode on the same ledger. Defaults are a 25 lb bag and a 60 second clock. The planner is exact branch-and-bound, deterministic, with a cap of 40 candidates and a node budget of 250,000. For a chosen set (S):

$$ \max_{S} \sum_{i \in S} v_i \ \sum_{i \in S} w_i \le 25,\quad \sum_{i \in S} t_i \le T,\quad \sum_{i \in S} r_i \le R $$

(v_i) is the appraisal. (w_i) is the vision weight, or a category default. (t_i) and (r_i) are always category defaults, never a measurement. Ties break to lower risk, then less time, then less weight.

Mode Risk budget (R) Per-item cap
Small job 6 nothing over 2
Big score 20 none

Exits are never treated as loot. The report also prints the same case at (T \in {30, 60, 120}) seconds under a 25 lb bag and 8 risk points.

Nothing here models getting in.

Voice

A second button, GPIO27, records about 4 seconds. Whisper transcribes it, then the answer is parsed from the live plan and the ledger. The recording is deleted after transcription. One call at a time. Bundled narrator clips cover the state changes, so those lines do not need a network.

How we built it

The hub is Python 3.13, Flask, and Waitress, with OpenCV on the camera path. State is one JSON ledger on disk plus the stills beside it. Optional Tiger Data stores a hypertable with a 5-minute aggregate, 7-day compression, and a non-blocking queue. The live case still runs from the JSON file if Tiger is off. Vultr serverless inference is a third vision provider, and VULTR_VISION_FIRST=1 puts it first. The public demo is a Docker image on Railway, health-checked at /health.

Vision order:

  1. OpenAI gpt-4.1
  2. OpenAI gpt-4.1-mini
  3. Anthropic claude-sonnet-4-6
  4. Vultr, only if a key is set

Speech is whisper-1. Spoken dispatch is tts-1, voice onyx.

The wrist firmware is an ESP32-WROOM-32 with an SSD1306 on I2C (SDA 21, SCL 22). It long-polls, so the screen is not stuck on a fixed refresh. A second wrist path, a Pi Zero with a Whisplay HAT, subscribes over MQTT. The rover is a Pi 4 with the same vision code, or ROVER_VISION=0 hands the frame to the hub.

What was hard

The interesting bugs were all about not lying.

  • A single flicker frame should not spend a vision call, so the scene gate waits for two confirms.
  • Two sightings of the same lamp should be one exhibit. A back door and a side door should not collapse.
  • A sold-listing search for a model must not price every pair of headphones on eBay, so comps require shared model tokens.
  • A campus id and a credit card look similar in a photo, so only Luhn-valid payment numbers are masked.
  • The reveal button must not erase the walk if the report fails to land on disk.

The boards are built. Motor, mic, pouch OLED, and ESP32 flash are in the repo as bring-up, with the software paths tested against mocks, including the 3 second drive cap, the queue of 50, and the report write. LiDAR was cut. Return-to-home still drifts if the wheels slip and the home tag is out of view.

What it is not

Use it in a permitted space. Visibility is not proof that something is unsecured. The kit does not test entry, bypass a lock, scan for vulnerabilities, or tell you what someone would take. The noir interface is the RowdyHacks theme. The output is a case file.

Built With

Share this project:

Updates

Submission history