Building an Agentic Enterprise  ·  Chapter 11 of 21
Chapter 11

Manage: Incidents, Rollback, the Off-Switch

What you do when something goes wrong, written down before something goes wrong

5min
max time to activate a kill-switch under CSA recommendations
4options
for risk response: mitigate, transfer, avoid, accept
3rehearsals
before treating any incident plan as real
Save PDF

Manage is the function whose value only becomes obvious during the incident. The teams that look composed during an agent failure rehearsed the response months earlier. The teams that look panicked are reading their own runbook for the first time.

The first incident

Every production agent will have an incident. The interesting question is not whether, but what shape. The most common shapes are: a confabulation that reaches a customer (Air Canada's bereavement-fare ruling is the canonical example); an indirect prompt injection that exfiltrates data (zero-click cases against Microsoft 365 Copilot, CVE-2025-32711); a tool-call cascade that takes irreversible action; a degradation that nobody notices until users complain.

The Manage function is the answer to the question: what is the first thing that happens, and who decides? NIST's Manage 1.3 codifies four risk-response options — mitigate, transfer, avoid, accept — but in practice, during an incident, those reduce to two: stop the agent, or constrain it. The choice depends on whether the agent is causing harm now (stop) or has caused harm and might compound it (constrain). The decision should not be made for the first time during the incident.

Rollback as a first-class operation

Most enterprise software has rollback as a deployment feature. Agentic systems need rollback as an operational feature: not just "deploy the previous version of the code", but "revert the agent to a known-good state, including its memory, its tool permissions, and any in-flight workflows."

Rollback for agents has three layers. The first is the agent itself — its prompt, its model, its tool registry. This is the easy layer; standard CI/CD handles it. The second is in-flight work: tasks the agent is currently executing, queues it has accepted, customers it has begun to respond to. These need to be paused, drained to humans, or rolled back to a pre-agent process. The third is downstream artefacts: records the agent has written, communications it has sent, transactions it has triggered. Some of these are reversible; some are not.

The discipline is to know, before deployment, which of those three layers your incident plan addresses. If the answer is only the first, your rollback is software, not operations.

The kill-switch

The CSA Agentic RMF Profile recommends "pre-authorized automatic containment responses — including automated agent suspension or kill-switch activation — for the highest-severity incident patterns". In plain language: a button that any on-call engineer can press, which suspends the agent's ability to act, revokes its credentials, and halts in-progress tasks, in under five minutes.

The kill-switch is unglamorous and easy to under-build. The version that works has all of the following: a single command or button that any on-call can invoke; pre-revocation of credentials at the IAM layer (not just disabling the agent's code); a way to drain in-flight work to humans; logging so the action is reconstructible; and — critically — a rehearsal cadence. The kill-switch that has never been tested is not a kill-switch. It is a hope.

Flywheel note

Every incident, however small, is fuel for the golden set, the eval suite, and the Map artefact. The teams that run a real flywheel treat incident review as a curation activity. The teams that don't are the teams that meet the same incident twice. Part III returns to the flywheel as the core organising idea of any agent program that compounds.

Communications, regulators, and the post-mortem

The hardest part of the Manage function is not technical. It is the conversation with the customer who was affected, the executive who has to approve the public statement, and (in regulated sectors) the regulator who needs to be notified. These conversations are easier when they have been rehearsed and harder when they have not.

The Air Canada case is instructive in this respect. The technical failure was a chatbot that promised a refund policy that did not exist. The expensive failure was the airline's subsequent argument that it was not responsible for what its chatbot said. The tribunal disagreed, and the cost of the case was less the refund itself than the legal precedent and the public coverage. A pre-written communications protocol — who speaks, what is acknowledged, what is committed — would have changed the outcome more than any technical fix.

Post-mortems for agent incidents should follow the same blameless format used elsewhere in modern operations, with one addition: every incident must produce at least one new entry in the golden set, and at least one update to the Map artefact. If neither happens, the Manage function has not closed the loop.

The next chapter is the artefact that makes all of this concrete: the agent registry, the single place where the inventory, the authority, and the kill-switch live.

Incident response, drawn Detect → decide (stop or constrain) → roll back three layers → post-mortem feeds Map and Measure. DETECT alert · CSAT drop red-team finding DECIDE stop · constrain KILL-SWITCH <5min · revoke creds drain in-flight CONSTRAIN narrow tools force HITL L1: agent code · prompt · tools L2: in-flight work · queues L3: downstream artefacts ROLLBACK · 3 layers POST-MORTEM → new golden-set case · update Map artefact
Figure 11.1Incident response for an agent: triage, decide (stop or constrain), rollback the three layers, and feed the post-mortem back into Map and Measure.