An Agent's Autonomy Should Stop Where Undo Does

Q: An Agent's Autonomy Should Stop Where Undo Does

An AI agent's autonomy should be capped by reversibility rather than scored by risk and value alone. Revoking an agent's permissions protects future actions but not ones already executed, so actions that cannot be undone, or that are visible outside the organisation, should be limited to preparation and human approval regardless of their value. Reversibility should be declared in each capability's contract by the capability owner, not estimated by the agent, and it usually has an expiry: a payment can be voided before settlement but only compensated after it. Reversible and compensable are different properties, since compensation corrects an outcome but leaves the original action and its effects in place. Because an approval assumes the action can still be undone, it should expire with the reversibility window and be checked again by a deterministic gate at the moment of execution.

Every autonomy model for AI agents I have seen draws the same ladder.

At the bottom, the agent suggests and a human decides. Then it prepares an action and a human approves it. Then it executes approved actions. At the top, it acts on its own within some delegated authority. Which rung an action sits on is decided by risk and value: an agent may schedule a meeting by itself, but it may not approve a loan.

It is a reasonable ladder, and it is sorted by the wrong thing. Risk and value tell you whether an action is worth taking and how much it matters if it goes wrong. They do not tell you the one property that decides how bad a mistake can actually get: whether it can be undone.

This is a design argument rather than a war story. It comes from applying what this series has been saying about reversibility to the systems everybody is now trying to give autonomy to.

Revocable is not the reassurance it sounds like

The reassurance that comes with almost every agent design is that autonomy is revocable. There is a kill switch. Permissions can be withdrawn at runtime. If the agent misbehaves, we stop it.

All true, and all about the future. Revoking an agent protects every action it has not taken yet and none of the ones it already has. The email to the customer has been sent. The record has been deleted. The announcement is public. Withdrawing the agent's permissions a minute later changes nothing about any of them.

So the question that decides how much freedom an agent can safely have is not whether we can stop it. It is whether we can contain what it did before we stopped it.

Reversibility is a ceiling, not a score

Once you look at actions that way, the ladder reorders itself.

A large action that can be cleanly undone can carry a lot of autonomy. Reallocating a budget line in a draft plan, provisioning a test environment, placing a refundable hold. If the agent gets it wrong, someone reverses it and the cost is an afternoon.

A small action that cannot be undone should carry very little, whatever its value. A two-line email to a customer. A deletion with no backup. A post on the company's account. Each is cheap in isolation, and each is permanent the moment it happens.

That is why I would not add reversibility to the autonomy score as one more weighted factor. A score lets a high value in one dimension compensate for a bad one in another, and irreversibility is exactly the property that should not be compensated for. It sets the ceiling. Risk and value can lower an action below that ceiling. Nothing should raise it above.

Declare it, don't let the agent judge it

If reversibility is the ceiling, the next question is who decides what is reversible.

Not the agent. If the agent classifies its own actions, the gate rests on the very judgment it exists to contain, and a confident misclassification is indistinguishable from a correct one until something irreversible has happened.

Reversibility belongs in the capability contract, declared by whoever owns the capability, next to the inputs, outputs and side effects it already defines. The owner of the payments service knows which operations can be voided. The owner of the messaging service knows that a sent message stays sent. That knowledge should be written down where a deterministic gate can read it, not reconstructed by a model at runtime.

Reversibility expires

Here is the part most autonomy models miss entirely, and it is the same thing this series found in architecture decisions: the cost of undoing something is not fixed at the moment of the decision. It grows afterwards.

A card payment is the clearest case. An authorisation can be voided before it is captured, and the customer barely notices. After capture and settlement, the money can only come back through a refund, which is a new transaction with its own trail. A message sitting in an outbox can be pulled. Once delivered, it cannot. A configuration change behind a flag can be switched off until something else starts depending on it.

So reversibility is not a flag. It is a flag with a window, and the window is set by the state of the domain, not by the agent. The contract has to say both: can this be undone, and until when.

That also changes how you read staging and compensation steps. A hold before a capture, a delay before a message goes out, a draft before a publish: these are not bureaucracy. They are how you buy a window. They turn an irreversible action into one that is reversible for a while, which is often enough for a human or a check to catch the mistake.

Reversible is not compensable

There is a temptation to treat any action with a compensating step as reversible. It is not.

A compensating action corrects the outcome. It does not erase the original. Refunding a settled payment restores the balance, but the charge happened, it appears on a statement, it may have triggered a fraud check or a loyalty update or an accounting entry downstream. A correction email does not unsend the first one. The history stays, and so does whatever reacted to it.

So the contract needs three values, not two. Reversible means it can be undone without trace while the window is open. Compensable means it can be corrected but leaves a history. Irreversible means neither. They deserve different ceilings, because they fail differently.

An approval is a claim about a state

The last gap shows up in the approval step itself.

A sensible design binds an approval to the exact action: the same target, the same parameters, the same intended outcome. If any of them changes after approval, the approval no longer applies. That is necessary, and it is not enough.

Consider an approval given at ten in the morning for a refund that can be voided until settlement at noon. The execution queue backs up. The action runs at half past twelve. Nothing about the action changed. Everything about its consequences did. The person who approved it approved a reversible action, and what ran was a compensable one.

An approval is not only saying "this action is fine". It is saying "this action is fine because it can still be undone". That is a claim about a state, and it should expire with the state. So the gate checks at commit, not only at approval. The approval carries an expiry, the earlier of its own lifetime and the end of the reversibility window. Immediately before execution, a deterministic gate re-reads the window from the domain, because the window may have moved, and refuses to proceed if it has closed. The action then goes back for approval under the rules for actions that can no longer be undone.

In a staged flow, that means the window is checked twice: once when the action is validated, and again at the moment it commits. Everything in between is time during which the world can change under an approval that still looks valid.

The code

Two small pieces make this concrete. The capability contract declares reversibility, and the ceiling is derived from it rather than scored.

// capability.ts
// Reversibility is declared by the capability owner, never estimated by the agent.

export type Reversibility =
  // Can be undone without trace, but only until a point in time set by the domain.
  | { kind: 'reversible'; undo: string }
  // Can be corrected afterwards, but the original action and its effects remain.
  | { kind: 'compensable'; compensate: string }
  | { kind: 'irreversible' }

export type Autonomy = 'suggest' | 'prepare' | 'execute-with-approval' | 'execute'

export interface Capability {
  name: string
  reversibility: Reversibility
  externallyVisible: boolean // anything a customer, partner or regulator can see
}

const ORDER: Autonomy[] = ['suggest', 'prepare', 'execute-with-approval', 'execute']

// The ceiling is set by what can be undone. Risk and value can lower it, never raise it.
export function ceiling(cap: Capability): Autonomy {
  if (cap.reversibility.kind === 'irreversible' || cap.externallyVisible) return 'prepare'
  if (cap.reversibility.kind === 'compensable') return 'execute-with-approval'
  return 'execute'
}

export function allowed(cap: Capability, requested: Autonomy): Autonomy {
  const max = ceiling(cap)
  return ORDER.indexOf(requested) <= ORDER.indexOf(max) ? requested : max
}

The approval binds to the exact action and to time, and the gate checks it again immediately before execution. It reads the window from the domain at both points, never from the agent.

// approval.ts
// An approval is a claim about a state: "this is fine because it can still be undone".
// It expires with that state, so it is checked again at commit.
import { createHash } from 'node:crypto'

export interface StagedAction {
  capability: string
  target: string
  params: Record<string, unknown>
}

export interface Approval {
  actionHash: string
  approvedBy: string
  expiresAt: number // the earlier of the approval's own TTL and the reversibility window
}

const APPROVAL_TTL_MS = 15 * 60 * 1000

export const hashAction = (a: StagedAction) =>
  createHash('sha256')
    .update(JSON.stringify([a.capability, a.target, Object.entries(a.params).sort()]))
    .digest('hex')

export function approve(
  action: StagedAction,
  approvedBy: string,
  reversibleUntil: number, // read from the domain at validate time
  now = Date.now()
): Approval {
  if (reversibleUntil <= now) throw new Error('window already closed: approve as irreversible')
  return {
    actionHash: hashAction(action),
    approvedBy,
    expiresAt: Math.min(now + APPROVAL_TTL_MS, reversibleUntil),
  }
}

export type CommitDecision =
  | { ok: true }
  | { ok: false; reason: 'action-changed' | 'approval-expired' | 'window-closed' }

// Called by the deterministic gate immediately before execution, never by the agent.
export function checkAtCommit(
  action: StagedAction,
  approval: Approval,
  reversibleUntil: number, // read from the domain AGAIN: the window may have moved
  now = Date.now()
): CommitDecision {
  if (hashAction(action) !== approval.actionHash) return { ok: false, reason: 'action-changed' }
  if (now >= reversibleUntil) return { ok: false, reason: 'window-closed' }
  if (now >= approval.expiresAt) return { ok: false, reason: 'approval-expired' }
  return { ok: true }
}

Note what the agent never does here. It does not decide what is reversible, it does not issue approvals, and it does not run the commit check. It stages actions. Everything that determines whether they happen sits in code the agent cannot talk its way past.

There is a ceiling here too

This can be overdone, and overdoing it has a specific failure mode.

If every action needs a human approval, humans stop reading approvals. A queue of forty identical requests a day trains people to click yes, and an approval nobody reads is worse than none, because it moves the accountability onto someone who never really decided. The same thing happens to any renewal that lands in a queue instead of on a person.

So the point is not to gate everything. It is to make sure the gates sit where undo stops, and nowhere else. Let the agent act freely inside windows that are genuinely open, keep humans for the moments when a window is about to close, and spend their attention where it cannot be bought back.

The rule I'd leave you with

Risk and value decide whether an action is worth taking. Reversibility decides how much freedom an agent can have while taking it. Declare it in the contract, give it a window, keep compensable apart from reversible, and treat every approval as a claim that expires with the state it was given in.

So here is the question I would ask of any agent you are about to give more autonomy to. Which of the actions it can take today could nobody in your organisation undo by tomorrow morning, and who approved each of them, and when?