Agents need a control plane before they need more autonomy.

The central idea: an agent is not autonomous because it can call tools. It is operationally autonomous because the surrounding system can constrain, observe, interrupt, and account for what it does.

Autonomy is a system property

Many agent demonstrations combine a model, a loop, and a broad set of tools. That can be effective for exploration, but production autonomy is much narrower. The system must know the task, authority, budget, state, and stop condition for every run.

That surrounding design is the control plane. It turns an unconstrained sequence of model decisions into an accountable operating process.

1. Tools must be narrow and business-shaped

A tool interface should represent a real operating action, not expose a database or an entire API. Prefer a tool such as search_customer_orders or draft_fulfilment_plan over a generic “query all records” capability.

For every tool, define:

• authorized inputs and actor;
• valid states and preconditions;
• read, draft, or write classification;
• idempotency and duplicate protection;
• validation and typed errors;
• reversibility or compensation;
• audit record and trace context.

2. State must live outside the conversation

An agent run is a process, not a transcript. Long-running work needs durable state: current phase, completed actions, pending approvals, tool results, deadlines, budget consumption, and failure information.

Persist state in a system designed for the workflow. The model can propose the next state transition, but the runtime should validate and own the transition.

3. Identity must follow the run

Tool calls should execute with a known service or user identity and an explicit context. Do not give the model a permanent administrator credential because it is simpler to implement.

Carry the user, tenant, purpose, delegation, and policy context through the run. This enables least-privilege authorization and meaningful audit records.

4. Human gates belong in the architecture

Not every decision needs human review. A useful gate is placed where uncertainty, consequence, reversibility, or policy justify it.

An approval object should capture:

• the proposed action and arguments;
• the evidence and assumptions used;
• risk classification;
• the approver role;
• the decision, reason, and timestamp;
• expiry and revalidation rules.

5. Budgets and stop conditions prevent open-ended work

Agents need explicit constraints on time, token or tool spend, number of steps, data access, and repeated actions. A run should stop when the objective is complete, evidence is insufficient, policy is exceeded, a dependency fails, or a human decision is required.

The stop condition should be observable and distinguishable from a crash or timeout.

6. Traces explain behavior, not just tokens

A useful trace records the task, actor, policy decision, evidence used, tool calls, state transitions, approvals, outputs, costs, errors, and final status. It should be possible for an operator to answer: what did the agent believe, what did it do, and what changed because of it?

7. Recovery is part of the agent design

Tool calls can succeed while the response is lost. A workflow can pause after an approval. A downstream system can accept a write but fail to publish an event. The runtime needs idempotency, resumability, compensation, and a queue of interrupted work.

Do not solve recovery by asking the model to “try again.” Recovery is a state-machine and integration problem.

8. Evaluate end-to-end task success

Evaluate whether the system completed the intended task correctly, within policy, at an acceptable cost and latency, without duplicate impact. Also include cases where the correct behaviour is to refuse, ask, defer, or choose a lower-risk path.

A production agent checklist

1. Is every tool narrow, validated, and classified by impact?
2. Is the durable state explicit and owned by the runtime?
3. Can every action be attributed to an actor and purpose?
4. Are human gates placed by consequence and uncertainty?
5. Are budget, timeout, and stop conditions enforced?
6. Can an operator trace and interrupt a run?
7. Can the system resume, compensate, or safely repeat?
8. Does evaluation include refusal and escalation cases?
9. Are model, prompt, tool, and policy changes controlled?
10. Is there a clear owner for failures and outcomes?

More autonomy should be earned

The safest path is not to avoid agentic systems. It is to expand authority based on evidence. Start with a read or draft path, observe the real failure modes, improve the control plane, then authorize the next bounded action.

Autonomy is not the absence of a human. It is a deliberate distribution of decision authority backed by architecture, evidence, and accountability.