Why your AI incident playbook can't wait for the first outage

This week's headlines covered observability in procurement, gateway prompt-injection filters, SOC 2 AI addenda, and contract fights over silent model updates. The operational thread is the same: agents are production systems now, and production systems need incident response the same way your servers and payment flows do.

Here is a practical AI incident playbook for a mid-size operator who does not have a dedicated AI SRE team yet.

Why AI incidents are not classic outages

Traditional incidents are often binary: service down, database unreachable, certificate expired. Agent incidents are behavioral:

• Wrong answer shipped to a customer with confidence.
• Tool called twice, duplicate charge or duplicate ticket.
• Runaway loop until token budget or rate limit trips.
• Sensitive data echoed in a response or logged in plain text.
• Model update changed refusal behavior on a regulated workflow.

Your playbook needs severity tiers for quality and trust, not just availability.

Define severity tiers (P1-P3)

Keep it simple. Three tiers are enough to start:

• P1: confirmed data leak, financial harm, or regulatory reportable event. Page security and legal immediately. Disable affected agent or tool globally.
• P2: widespread bad outputs, runaway spend above threshold, or customer-facing workflow broken for many users. Page on-call engineering. Roll back prompt or model pin.
• P3: isolated bad run, elevated error rate, or observability gap discovered. Ticket for next business day unless trend worsens.

Write examples for your business. "Agent quoted wrong price on 3 orders" might be P2 for e-commerce and P1 if contracts are binding.

Containment steps (do these in order)

1. Stop the bleeding: disable tool, feature flag off agent, or switch to human-only mode for the affected workflow.
2. Preserve evidence: export traces, prompts, tool inputs/outputs, and model version IDs before logs rotate.
3. Assess blast radius: how many users, which data classes, any downstream actions taken (emails sent, tickets created, payments).
4. Fix forward: roll back config, add golden-task regression, re-enable with tighter caps.
5. Communicate: internal status first, then customer notice if impact warrants it (use pre-approved templates).

Containment beats root cause analysis in the first hour. You can explain why later.

Roles and who gets paged

Minimum roster for a P1/P2 AI incident:

• Incident commander: owns timeline and decisions.
• Agent owner: knows prompt, tools, and deployment path.
• Security/privacy: data classification and breach assessment.
• Comms: customer and executive messaging.
• Vendor liaison: if SaaS platform involved, open ticket with trace IDs.

For P3, a single agent owner plus ticket queue is fine. Do not page everyone for every hallucination.

What to log before you need it

Incidents fail when teams cannot reconstruct a run. Minimum observability:

• Correlation ID per user request through all agent steps.
• Model ID and prompt version hash at time of run.
• Each tool call: name, arguments (redacted), result status, latency.
• Token spend and step count per request.
• Human override or approval events.

If your vendor cannot export this, that is a procurement problem, not an ops problem you solve after the fact.

Customer communication templates

Pre-write three short templates:

• Acknowledgment: we are investigating an issue with [workflow], impact may include [X], we have disabled [Y].
• Resolution: root cause summary in plain language, what we changed, what customers should do if affected.
• No customer impact: internal issue only, no action needed.

Legal should review once. Do not draft under pressure at midnight.

Post-incident: add a golden task

Every P1/P2 closes with one new item in your production eval suite: a task that would have caught this failure before re-enable. Tie playbook closure to eval pass rate, not just "we fixed the prompt."

Common mistakes

• No severity definitions, so every bad run feels like an emergency.
• Deleting logs to "clean up" before evidence is preserved.
• Announcing root cause before blast radius is known.
• Playbook lives in a wiki nobody has read; run a tabletop once a quarter.

A two-week starter plan

Week 1: Write P1-P3 definitions with three real examples from your workflows. Assign incident commander and agent owner. Confirm trace export works from your platform.

Week 2: 30-minute tabletop: "agent double-charged a customer." Walk containment steps. File gaps as tickets. Publish one-page playbook link in the on-call runbook.

Bottom line

AI incident response is not pessimism. It is how you keep autonomy shippable when models, prompts, and tools change weekly. Observability, severity tiers, and pre-written comms turn a Friday night surprise into a managed event your board can understand. The natural follow-on is human approval queues for the actions your playbook says must never auto-run.

Questions on incident playbooks for your agents? Reach out and we will point you at the right pattern.

Continue reading: latest headlines on approval queues and board KPIs

Matt Potter · Swift Media