Adaptive SafeCommit

Adaptive verification and automatic rollback for coding agents: SafeCommit checkpoints changes, scores risk, independently verifies them, and restores the workspace when unsafe.

Comment

Inspiration

Autonomous coding agents can successfully complete a task while still leaving the workspace in an unsafe state.

That distinction became the core idea behind Adaptive SafeCommit:

Task completion does not necessarily mean trusted state.

A coding agent may correctly follow an operator's instruction, modify deployment configuration, update application code, or change dependencies — but the platform still needs an independent way to decide whether those changes should become durable.

We wanted to build that missing transaction boundary.

What it does

Adaptive SafeCommit treats every Agent-generated workspace change as speculative until it has been assessed and, when necessary, independently verified.

For every Agent run, SafeCommit:

  1. Checkpoints the workspace before execution.
  2. Lets the Agent complete the requested task normally.
  3. Computes the actual changed-file diff.
  4. Applies deterministic, explainable risk scoring.
  5. Detects available project verification capabilities.
  6. Selects a verification tier based on risk.
  7. Runs executable checks inside an independent verifier.
  8. Keeps successful candidates or rolls back rejected ones.
  9. Displays the decision and evidence directly in the Agent Playground.

The verification policy is adaptive:

  • Low risk — structural screening only, with 0 executable checks.
  • Medium risk — one targeted executable check, preferring tests.
  • High risk — all available checks such as typecheck, test, and build.

High-risk changes fail closed if sufficient verification is unavailable.

How we built it

Adaptive SafeCommit extends the provided Volc Agent Launchpad while preserving its existing Agent CRUD, browser Playground, persistent workspaces, Codex CLI runtime, and Ark model integration.

The middleware is implemented around the normal Agent run lifecycle.

Transactional workspace recovery

Before execution, SafeCommit creates a checkpoint using a control-plane-owned bare Git repository.

The Git metadata is kept outside the Agent workspace. If verification later rejects the candidate, SafeCommit restores tracked files and removes newly-created files, returning the workspace to its pre-run state.

Deterministic risk engine

After Agent execution, SafeCommit examines the files that actually changed.

Risk signals include:

  • executable source changes;
  • configuration changes;
  • dependency manifests;
  • security-sensitive paths;
  • persistence/database paths;
  • infrastructure and deployment files;
  • larger change sets.

The resulting LOW / MEDIUM / HIGH score is deterministic and explainable to the user.

Adaptive verification

SafeCommit detects npm project capabilities from package.json, including:

  • typecheck
  • test
  • build

Verification effort then scales with structural risk rather than running every possible check after every Agent edit.

Independent verifier

Executable checks run in a separate container with a narrower trust boundary than the Agent runtime.

The verifier has:

  • no model credentials;
  • no external network access;
  • dropped Linux capabilities;
  • no-new-privileges;
  • a read-only root filesystem;
  • a read-only view of the authoritative candidate source;
  • an ephemeral writable tmpfs workspace;
  • CPU, memory, PID, timeout, and output limits.

The Agent therefore does not grade its own work.

Evidence UI

We extended the Playground with a SafeCommit evidence card showing:

  • risk score and level;
  • reasons for the classification;
  • changed files;
  • selected verification policy;
  • planned versus executed checks;
  • pass/fail status and duration;
  • accepted, rejected, blocked, or rolled-back outcome.

This makes the middleware decision visible instead of hiding it inside backend logs.

Challenges we ran into

One of the most important challenges was designing verification so that it could not mutate the same workspace it was judging.

Our first verifier design mounted the Agent workspace directly into the verification container. Even though the verifier was independent from the Agent, a test script could still modify the candidate being evaluated.

We changed the design so that:

  1. the authoritative workspace is mounted read-only;
  2. its contents are copied into an ephemeral tmpfs workspace;
  3. verification executes only against that disposable copy.

We validated this with a real Docker integration test that deliberately writes a side-effect file during verification and confirms that the file never appears in the authoritative workspace.

Another challenge was balancing safety against verification cost. Running the entire test/build pipeline after every documentation or trivial change would be wasteful, which motivated the adaptive LOW / MEDIUM / HIGH policy.

Accomplishments that we're proud of

  • Integrated SafeCommit into the normal Agent execution path instead of creating a separate demonstration endpoint.
  • Implemented automatic rollback for failed, cancelled, and verification-rejected Agent changes.
  • Built deterministic, explainable risk classification.
  • Added risk-adaptive verification routing.
  • Created an isolated, credential-free verifier.
  • Preserved SafeCommit evidence even when a candidate is rejected.
  • Built a visual Evidence UI that makes the middleware decision immediately understandable.
  • Added automated coverage for low-, medium-, and high-risk behavior.
  • Verified the isolation model using a real Docker smoke test.

What we learned

The most important lesson was that Agent safety is not only about constraining what the model can do.

There is also a system-design question:

Who decides whether the resulting state should be trusted?

Separating Agent execution from independent verification creates a much stronger control boundary than asking the Agent to validate its own output.

We also found that safety and efficiency do not have to be opposites. Deterministic structural signals can cheaply route changes toward an appropriate verification budget instead of treating every mutation identically.

What's next for Adaptive SafeCommit

Potential extensions include:

  • semantic risk analysis for structurally ambiguous changes;
  • support for more package managers and language-native test systems;
  • configurable organization-specific risk policies;
  • crash-time rollback after unexpected control-plane restarts;
  • richer verification-cost and rollback telemetry;
  • integration with deployment approval and CI/CD systems.

Our longer-term goal is to make SafeCommit a reusable transaction layer for autonomous software and infrastructure agents.

Adaptive SafeCommit gives Agent platforms a transaction boundary between task completion and trusted state.

Built With

Share this project:

Updates

Submission history