Counterstep

Counterstep safely reverses an AI agent’s overreach and proves exactly what it repaired.

Comment

Inspiration

Counterstep grew out of a problem I noticed while working on Agent Receipt, an earlier project that reconstructs an agent run and checks it against the authority the user gave it. The receipt could show exactly where an agent went too far, but the operator still had to clean up the consequences.

Imagine that an agent finishes a churn-analysis task, but leaves restricted customer data in an externally shared spreadsheet and queues an email that nobody approved. Knowing what happened is useful. The spreadsheet is still exposed, though, and the message is still waiting to be sent.

I built Counterstep to handle that next part. It starts after a run is complete, checks what is true now, repairs whatever is still reversible, and leaves evidence behind for every decision and action.

What it does

Counterstep takes an evidence-linked receipt for a completed agent run and turns it into a tightly bounded recovery job. It inspects the affected resources, gives Gemini a small set of recovery tools through Google ADK, and checks the proposed plan before any write is allowed.

The demo has four scenarios that a judge can choose before starting the run:

  • Canonical recovery: revoke external access to the spreadsheet and cancel the queued message.
  • Already safe: inspect both resources and leave them alone because no repair is needed.
  • Irreversible delivery: repair the spreadsheet, but report that a delivered message cannot be recalled.
  • Stale-state replan: reject a write after the resource changes, inspect everything again, and admit one replacement plan without overwriting the newer state.

Every write has to point back to an approved plan step and a known event or policy finding. It also needs the exact version that was inspected, an allowed state transition, remaining remediation authority, and an idempotency key created by the application. Gemini proposes the plan, but ordinary deterministic code decides whether that plan is allowed to run.

After the recovery attempt, Counterstep reads the resources again. Those fresh snapshots and the recorded tool results determine the final outcome. The model does not get to declare its own success. The run ends with an action receipt, a qualified closure receipt, and a SHA-256 digest that can be checked independently.

How I built it

Counterstep is a strict TypeScript application built with Next.js 16.3.3, React 19, TypeScript 6, and Zod 4. The browser talks to server routes, and the server owns the incident, authority, run, and closure state.

For the live model path, Gemini 3.5 Flash Lite runs on Vertex AI through the Google Agent Development Kit for TypeScript. I gave it five narrow tools: inspect a resource, submit a recovery plan, revoke spreadsheet access, cancel a queued message, and request closure verification. Gemini only receives minimized incident facts and the sandbox state that Counterstep has actually inspected.

The deterministic side of the application handles policy evaluation, citations, plan admission, resource versions, transition rules, idempotency, write budgets, action-receipt verdicts, and closure. I repeated the most important checks inside the repository transaction because an approved plan can still become unsafe if the underlying resource changes before the write lands.

Firestore stores the recorded live-evidence runs, including resource versions, inspections, plans, events, idempotency records, admission counters, and closure receipts. I packaged the app with Cloud Build and Artifact Registry, then deployed it to Cloud Run with separate build and runtime service accounts.

The public demo runs the real server-side recovery workflow on Cloud Run, but uses a deterministic planner and in-memory synthetic state. This lets judges exercise the complete gate, tool, event, scenario, and closure flow without spending Gemini or Firestore quota. The video and repository contain the separate evidence from the live Gemini, Google ADK, Vertex AI, and managed Firestore runs.

Challenges I faced

The central design problem was deciding what the model should control. Gemini needs enough freedom to inspect the incident and choose a useful repair, but it should never be able to expand its own authority. I ended up treating the model's output as a candidate plan rather than a command. Counterstep checks that candidate against the receipt, the remediation authority, and the latest inspected state.

Stale state was the hardest edge case. Retrying a rejected write with a newer version would quietly bypass the original approval. Counterstep instead re-inspects every governed resource, creates a new candidate plan, runs that plan through the gate, and preserves both plans in the event history. If the replacement becomes stale too, the run stops.

Cloud deployment also produced a useful failure. My first Developer API attempt hit depleted prepaid credits. Counterstep stopped before inspection and recorded zero writes. I kept that failure, moved the live deployment to Vertex AI workload identity, and ran the evidence journey again. It was a good test of the fail-closed behavior because the system did exactly what it was supposed to do when the model was unavailable.

I also had to be careful about evidence. A fixture run, a Firestore emulator test, a managed-cloud test, a live model run, and a screenshot prove different things. The repository labels them separately so a passing local test is never presented as proof that an untested cloud path worked.

What I learned

The agent became easier to trust as I narrowed its freedom. Five small tools were enough for the recovery job. The model could still decide what needed attention, while the rest of the system remained predictable and testable.

I also learned that recovery needs a higher standard of proof than task execution. A convincing model response is not evidence that a spreadsheet permission changed or that a message was cancelled. Recorded tool results, exact versions, final snapshots, and an independently computed closure result are much more useful to an operator.

The project changed how I think about agent observability. Reconstructing an overreach is only half of the operator's job. The system should also help with the repair, but only inside authority that was defined before the recovery began.

What I am proud of

  • All four Recovery Test Rack scenarios use the same service and authority boundary.
  • The stale-state scenario produces a real version conflict, refuses the write, and completes one fully cited replan.
  • The irreversible-delivery scenario closes as partially_repaired instead of pretending that a sent message was recalled.
  • The release passes 445 automated tests, five deterministic evaluation cases, strict TypeScript, ESLint, a production build, and release and privacy audits.
  • Eight production-repository cases pass against the official Firestore emulator, and six retained adapter cases pass against managed Firestore.
  • Three live Gemini and Google ADK journeys completed with two authorized writes, 12 accounted events, and digest-valid closure receipts.

Reuse disclosure

Counterstep was built during the hackathon on top of a disclosed Agent Receipt foundation. That earlier foundation supplies exact-byte trace hashing, canonical event accounting, deterministic policy findings, the original receipt, and the synthetic CRM fixture.

The recovery system is new hackathon work. This includes the remediation authority, Gemini and Google ADK orchestration, five recovery tools, deterministic plan and write gates, transactional Firestore repository, stale-state replan, Recovery Test Rack, closure receipt, cloud deployment, and the related tests.

Links

Built With

Share this project:

Updates

Submission history