Handback Agent

An AI mediator that knows when not to decide. It keeps helping two people resolve a conflict — and returns the decisions that belong to them.

Comment

Inspiration

AI agents are increasingly asked to help with decisions that matter: who keeps how much equity, what a license is worth, how much of a deposit comes back. A capable model will usually produce an answer when asked — even when the next step is not its call. Refusing outright isn't the fix either: it stops being useful the moment the conversation gets hard.

We wanted an agent that keeps helping right up to the boundary of its role, recognizes when a request would make it the decision-maker, returns that decision to the people, and then picks the work back up from what they decided.

What it does

Handback Agent mediates a conflict between two people, A and B. You pick a scenario — cofounder equity, illustration rights, or a rental security deposit — and play either side.

  • It continues within its role. It checks both positions and offers nonbinding suggestions (e.g. 35% between a 40% ask and a 30% offer). Nothing is decided until A and B accept.
  • It returns decisions it should not make. When B asks "AI, please set the final equity share," the agent stops and explains why: "Following this request would make the AI the judge or final decision-maker. So the AI returns this decision to the people." It lists the ways A and B could decide, and does not recommend any of them.
  • It resumes from the people's decision. Their choice is recorded under Decided by the people, checked again, and the agent moves on to the next step instead of repeating its earlier suggestion.
  • It blocks unsafe requests separately. Asking for its hidden instructions, other people's data, or a rigged outcome is stopped by a security gate before the authority check runs.

A side panel shows each turn's path — Security → Authority → Response → Record — so you can see why the agent did what it did.

How we built it

Every turn runs through a small pipeline. Each stage is a runtime call to NVIDIA Nemotron 3 Ultra (nvidia/Nemotron-3-Ultra-550b-a55b) on Nebius Token Factory, in JSON-object mode:

  1. Security Gate — is this message safe to process at all? (PASS / SECURITY_BLOCK)
  2. Authority check — does the next step belong to the AI? (CONTINUE / HANDBACK, with a reason code such as role transition or third-party rights)
  3. Response — on CONTINUE, the next mediation move in plain language; on HANDBACK, a brief that names the decision, 2–3 options for the people, and the condition for resuming
  4. Record — a positions table updated only from what the parties actually said

We split the reasoning budget by stage: Nemotron's thinking is turned on only for the authority check, where the hard judgment happens, and off for the gate and the wording stages to keep the demo responsive. Every response is validated against a strict schema in code; an invalid response is retried once and otherwise fails closed, so the model can't quietly change the contract. We evaluated the Security Gate and the authority check on the same fixed test sets in Korean and English, and their decisions matched across both languages.

The same pipeline runs locally in Python and as a Cloudflare Worker for the public demo. Prompts, scenarios and screen text are generated from the Python sources, the API key stays a Worker secret, and conversation state stays in the visitor's browser.

Across 252 demo turns (581 model calls), a turn took a median of 5.3 seconds and about $0.004 in model cost.

Challenges we ran into

1. Resuming without bouncing straight back to the people. The first version of resume simply sent the original request back through the pipeline after the people decided. The authority check saw "AI, set the final share" again and returned the decision again — a handback loop. We changed what resume carries: the people's choice is recorded as an item already decided by the authorized people, the original request is described as settled by them, and their decision itself passes the Security Gate again before anything continues. That surfaced two subtler problems. The agent sometimes reopened what the people had just decided as a fresh "suggestion," so asking the AI to re-decide something the people already decided now counts as a handback too. And after resuming, it tended to repeat its earlier 35% suggestion, which made "resume" look like "repeat" — so resume also tells it that the earlier suggestion is already on the table, and it moves on to the next step, such as asking each side for the reasons behind their number.

2. Turning an authority classifier into a working agent. Our authority check (we call it the CSM) started as a research classifier: one-line case descriptions in, five fixed fields out (candidate action, decision, reason, failed gate, basis). A product needs more than a verdict. Rather than expanding that contract — which would have meant re-validating it from scratch — we kept it fixed and added a separate Handback Brief call that only runs on a handback and writes the decision request, the options for the people, and the resume condition. Real conversations also aren't one line, so we built a state summary the CSM reads every turn: the goal, agreed facts, what the people already decided, open issues, each side's positions, and the AI's last question. Live use exposed mixed messages — a normal reply with a request to rule or to leak data tucked inside — so the rules now give the handback or the block priority over the harmless part. Finally, Nemotron can return a different reason code for the same input even at temperature 0, so the decision shape is enforced in code instead of trusting any single run, and we did not patch individual cases to make results look stable.

Accomplishments that we're proud of

  • Handback and resume work end to end on a live model, not as a scripted demo.
  • The agent can say why it is returning a decision, in one sentence a non-expert understands.
  • Security and authority are separate checks: a safe request can still be the wrong one for the AI to decide.

What we learned

The useful line isn't "can the AI answer this?" but "should this answer come from the AI?" Making that line explicit — as a check with reasons, not a refusal — is what lets an agent stay helpful and still leave consequential choices with people.

What's next

  • Evaluate the authority check on full multi-turn conversations, not only on single-line test cases.
  • Let the people delegate specific decisions to the agent explicitly, with that permission recorded and revocable.
  • Bring the same handback pattern to other agent workflows where an AI acts on someone's behalf.

Built With

Share this project:

Updates

Submission history