What Agentic AI Needs Before It Goes Anywhere Near Production

An agentic system differs from a chatbot in one important respect: it takes actions across multiple steps without a human turn between each one. That property is what makes it valuable, and it is also the entire source of its risk.

Most agentic pilots demonstrate the capability convincingly and then stall on the way to production. The reason is rarely the model. It is that the surrounding engineering — the part that makes autonomy operable — was never built.

Scoped, least-privilege tool access

An agent should be able to reach exactly the systems its task requires and nothing more. In practice this means dedicated credentials per agent role, read and write separated, and destructive operations either excluded entirely or gated behind explicit approval.

The common shortcut — giving the agent the same broad service account a human integration uses — turns a prompt injection or a reasoning error into an incident with an unbounded blast radius.

A supervisor that can actually stop things

Multi-agent systems need something above the agents that can arbitrate, enforce budgets, and terminate a run. Budgets matter in three dimensions: steps, wall-clock time, and cost. An agent that loops is not a hypothetical; it is a Tuesday.

• Maximum steps per run, after which the task escalates rather than continues.
• Time and token budgets enforced outside the agent’s own reasoning.
• A kill switch that operations can use without a deployment.
• Clear handoff contracts between agents, so a failure in one does not silently corrupt another’s input.

Tracing that survives the incident review

When an agent does something unexpected, the question is always the same: what did it actually do, and why? Answering it requires a persisted trace of the reasoning, the tool calls, the inputs and outputs of each, and the final state — retained long enough for the review that happens two weeks later.

This is not optional instrumentation you add once problems appear. Without it, every incident review ends in inference, and you cannot demonstrate to a risk function that the system behaves within its bounds.

Human checkpoints placed by consequence

The question is not whether to keep a human in the loop but where. Placing approval gates everywhere destroys the value of autonomy; placing them nowhere destroys the safety case.

Put the human where the cost of being wrong is high and the cost of waiting is low.

Irreversible actions, actions with financial or regulatory consequence, and actions where the agent’s own confidence is low are the usual candidates. Everything else runs, and gets reviewed in aggregate rather than case by case.

An evaluation harness that runs on every change

Agentic systems drift. A prompt change, a model version update, a modified tool description, or a change in the underlying data can all alter behaviour in ways that are invisible to spot-checking.

The mitigation is unglamorous and effective: a suite of representative tasks with known-good outcomes, run automatically against every change, with results compared to the previous baseline. It is the same discipline as regression testing, applied to a system whose output is not deterministic — which means the assertions check properties and outcomes rather than exact strings.

The honest summary

For well-bounded processes with good observability and a human escalation path, agentic systems are production-ready today, and we build them. For open-ended, high-consequence decisions with no review step, they are not — and no amount of prompt engineering changes that. Knowing which situation you are in is most of the work.