-
-
FixRepro shows the unsafe baseline, same-input patch rejection, and preserved trusted update.
-
Owner-controlled Roomba s9+ used as an audible physical-output witness during an authorized local proof of concept.
-
The complete dashboard connects exact-input hashes to the tamper-evident evidence digest.
-
The self-contained verification report records every device transition and verdict.
-
The full verification result remains usable on a mobile-width screen.
Inspiration
A security patch can look correct in code and still fail on the product.
That matters when software controls robots, smart devices, industrial equipment, and other connected systems. Before an update is released, a team needs to answer two practical questions:
- Is the unsafe behavior actually gone?
- Does the legitimate feature still work?
Today, that proof is often spread across pull requests, logs, screenshots, and manual tests. The code changed, but there is no single repeatable result that a security team, maintainer, auditor, or customer can verify.
FixRepro creates that missing security release gate.
What it does
FixRepro verifies a defined security fix through a three-part test:
- Reproduce the unsafe behavior on the vulnerable build.
- Replay the exact same input against the patched build.
- Run a trusted positive control to confirm that legitimate behavior still works.
In the demonstration, the vulnerable gateway accepts an integrity-valid software update from an untrusted signer. FixRepro then sends the exact same package bytes to the patched gateway. The patched build rejects them. Finally, a trusted update succeeds.
FixRepro returns PATCH_VERIFIED only when all three results are observed.
It then creates a tamper-evident evidence bundle containing the tested builds, package hashes, requests, responses, device state changes, individual verdicts, and a self-contained report. A separate verification command checks every bundle file and recomputes the final outcome instead of trusting a stored label.
What makes it different
FixRepro is not a vulnerability scanner and it does not generate patches.
It focuses on the missing step after a patch is written and before an update is released.
The exact-input replay matters because changing the test input would make it difficult to prove that the patch caused the different result.
The positive control matters because simply blocking every update would remove the vulnerability but also break the product.
The independent evidence check matters because a screenshot or report alone cannot prove that the underlying files remained unchanged.
The result is one repeatable release decision:
APPROVE, HOLD, or INCONCLUSIVE.
Physical-world proof of concept
I also tested an optional physical-output witness with an owner-controlled Roomba s9+.
During a separate authorized local test, the robot produced its supported Locate melody. This demonstrates how an accepted FixRepro result can be connected to a real physical output.
The Roomba did not receive firmware and was not the OTA test target. It did not move, start cleaning, change settings, or receive a firmware update.
The optional witness is disabled by default. The reproducible Docker workflow continues to use the synthetic virtual device. The physical witness cannot determine or change the core security verdict.
Who can use FixRepro
Robotics and IoT manufacturers
Verify a security fix before releasing an over-the-air software update to connected devices.
Product Security teams
Use a deterministic result as a release gate between patch approval and deployment.
Open-source maintainers
Verify that a contributed security patch blocks the reported behavior without breaking the supported feature.
Auditors and enterprise customers
Review exactly what was tested instead of relying only on a developer statement or screenshot.
Why it matters now
More physical products depend on software. More of those products receive remote updates. A single incorrect patch can therefore affect many connected devices.
Teams need a repeatable verification step between:
The patch was merged
and:
The update is safe to ship
FixRepro is designed to provide that step.
How it works
Each run starts from a controlled device state.
Vulnerable baseline
The virtual device begins at version 1.0.0.
The vulnerable gateway checks package integrity but does not enforce signer trust. It accepts the untrusted update, and the device changes to 9.9.0-test.
Result:
UNSAFE BEHAVIOR REPRODUCED
Patched security test
FixRepro resets the device and replays the exact same untrusted package bytes.
The patched gateway verifies the digital signature and trusted signer. It rejects the package, and the device remains at 1.0.0.
Result:
PATCH BLOCKED SAME INPUT
Positive control
FixRepro resets the device again and sends a trusted update.
The patched gateway accepts it, and the device changes to 1.1.0.
Result:
LEGITIMATE UPDATE PRESERVED
Only this complete sequence produces PATCH_VERIFIED.
How I built it
The project uses Python 3.12, FastAPI, Pydantic, HTTPX, and the cryptography library.
Ed25519 digital signatures protect the trusted update path. Private signing keys exist only in memory during package generation and are never written to disk.
Strict Pydantic models and JSON Schema validate the evidence format. The deterministic verdict engine calculates results from recorded behavior. The browser only presents those results and cannot override them.
The dashboard uses plain HTML, CSS, and JavaScript with no external frontend dependency.
Docker Compose starts the complete reproducible demonstration with one command. GitHub Actions runs the validators, automated tests, bundle checks, dashboard smoke tests, and Docker validation.
The optional Roomba witness uses a narrowly limited local MQTT adapter with one fixed Locate action. It provides no general robot-control interface and remains disabled during the standard Docker and CI workflows.
Challenges
The first challenge was proving that the vulnerable and patched builds received identical input. FixRepro reads the untrusted package once, holds one byte array in memory, sends it to both builds, and records matching envelope and request hashes.
The second challenge was avoiding circular hashing. An evidence file cannot contain its own final hash, and a report cannot contain a manifest digest that already includes that report. FixRepro uses a defined generation order and a separate manifest root digest.
The third challenge was preserving a strict browser boundary. The user interface can start only one fixed verification workflow. It cannot provide a target, package, key, command, file path, test case, or output directory.
The physical witness required the same separation. A robot signal can make a software result observable, but it cannot influence the underlying verdict.
Accomplishments
FixRepro now provides:
- A complete three-scenario security verification workflow
- Byte-for-byte same-input replay
- A trusted positive control
- Deterministic verdict calculation
- A tamper-evident evidence bundle
- Independent bundle verification
- A self-contained HTML report
- A responsive judge-facing dashboard
- One-command Docker startup
- Automated tests and GitHub Actions
- An optional physical-output-witness proof of concept
What I learned
A useful patch test needs more than a passing test case.
It needs an identical input, a confirmed initial state, observable consequences, stable build identities, and a positive control.
I also learned that evidence generation and evidence verification are different tasks. A readable report explains the result. An independent verifier establishes whether the recorded files and verdict remain consistent.
What is next
The next step is to add narrow adapters for authorized robotics, smart-device, industrial IoT, and connected-system test labs.
FixRepro could also be integrated into CI/CD and release-management systems so a security update cannot advance until the required regression, positive control, and evidence checks pass.
The goal is to turn:
We changed the code
into:
We proved the fix
Log in or sign up for Devpost to join the conversation.