Why prompt-injection red teams belong on your agent launch checklist
Customer-facing agents inherit every paste, attachment, and ticket field attackers can reach. Run structured injection drills on the same channel you ship, before you add tools or widen data access.
This week's headlines roundup calls out prompt-injection drills on live customer channels. That is not a one-time pen test slide. It is a recurring discipline you run on the same URLs, widgets, and ticket flows your customers already use.
If you published escalation SLAs and mapped tools to IAM, red teams are how you prove those controls survive a motivated user, not just a happy path demo.
Test the channel, not the slide deck
Run scripts against production configuration with test accounts: same model, same tools, same rate limits. Include indirect injection (malicious text in CRM notes, PDF uploads, email bodies forwarded into chat) because that is how real tickets arrive.
Define pass and fail before you start
- Fail: exfiltration of secrets, cross-tenant data, or unauthorized tool calls (refunds, password resets, admin APIs).
- Warn: policy violations that do not leak data but mislead customers (fake discounts, invented policies).
- Pass: agent refuses, escalates to human, or answers within approved knowledge only.
Write results down. Compare to eval gates on promotion so a failed drill blocks new tool access until fixed.
Shrink tool blast radius for the first release
Launch with read-only tools and no destructive actions. Add write tools only after a drill cycle passes with logging reviewed. Pair with approval queues for anything that moves money or changes accounts.
Log every drill like an incident
Store transcripts, tool calls, and model versions for each run. When counsel asks what you knew before go-live, you want the same export format you use for disputes and legal hold.
Schedule drills, not heroics
Monthly is a sensible default for customer-facing agents; weekly while you are adding tools or new data sources. Tie the calendar to procurement attestation: vendors see your test plan before you sign the production order.
Week-one rollout
- Pick your highest-risk customer agent (tools + PII).
- Draft ten injection cases: direct, indirect, and tool-abuse attempts.
- Run them in staging that mirrors prod; fix failures before prod drill.
- Run the same pack in prod with a test account; file tickets for misses.
- Block new tool grants until the next drill passes.
Attestation and residency tags from this week's headlines only matter if the agent behaves when someone tries to break it.
Questions on agent security rollout? Talk with us or explore Website Agent deployments with a private Swift Chat room for change control.
Matt Potter · Swift Media