CRM agent containment: scoped tools, dry-run, kill switch

It is 16:42 on a Tuesday. The operations lead at a forty-person Dutch logistics outfit is staring at a dashboard we built her, three days before we flip a sales-triage agent live against her HubSpot instance. The agent will read inbound replies, update deal stages, draft follow-ups, send the boring ones automatically. She has not asked us how clever it is. She has asked, twice, what stops it from emailing the wrong customer at two in the morning if something goes sideways.

That is the right question. Speed is the easy part. Containment is what gets an agent past the first board review and into production.

Anthropic, the team that trains Claude, published a useful engineering note on building effective agents that lines up with what we see in the field. The patterns translate almost directly to a CRM agent. We have been running three of them on every install for the past two years: scoped tools, dry-run mode, and a kill switch the ops lead can actually reach. The fourth pattern, budget caps, is one we picked up after the first time an unsupervised loop ran a four-figure API bill in an afternoon.

Scoped tools, not full API access

The mistake every first-time agent builder makes is to wrap the whole CRM SDK and let the model figure it out. It will figure it out. It will also figure out, on a quiet Sunday, how to merge two contact records because a prompt example was slightly ambiguous.

The fix is small surface area. Write the smallest tool that does the smallest useful thing. Five verbs that each do one job beat one tool that does eleven.

A triage agent we shipped to a SaaS client in March has four tools. That is the entire toolbox:

const tools = [
  {
    name: "get_deal",
    description: "Read one deal by id. No writes.",
    input_schema: {
      type: "object",
      properties: { deal_id: { type: "string" } },
      required: ["deal_id"]
    }
  },
  {
    name: "append_note",
    description: "Append a timestamped note. Cannot edit or delete existing notes.",
    input_schema: { /* ... */ }
  },
  {
    name: "set_stage",
    description: "Move a deal to a stage in the predefined pipeline. Refuses unknown stages.",
    input_schema: { /* ... */ }
  },
  {
    name: "schedule_follow_up",
    description: "Create a follow-up task for the deal owner. Does not send email.",
    input_schema: { /* ... */ }
  }
]

No delete. No merge. No execute_query. If the agent needs to do something we did not anticipate, it posts to a Slack channel and a human runs the action manually. We added one new tool in five months. The ops lead can read the full list in twenty seconds and tell you exactly what the agent can and cannot do. That clarity is the point.

Anthropic's own tool use documentation hammers the same line. Every tool is a hole in the wall, so cut as few holes as possible and shape each one to match exactly one verb.

Dry-run as the default, not the test

Every state-changing tool we write returns two different things depending on a flag.

def set_stage(deal_id: str, stage: str, dry_run: bool = True) -> dict:
    deal = crm.get_deal(deal_id)
    if dry_run:
        return {
            "would_have": f"set deal {deal_id} ({deal['name']}) "
                          f"from '{deal['stage']}' to '{stage}'",
            "wrote": False,
        }
    crm.update_deal(deal_id, stage=stage)
    audit.write(actor="agent", verb="set_stage",
                deal_id=deal_id, before=deal['stage'], after=stage)
    return {"wrote": True, "stage": stage}

On day one of any install, dry_run is hard-coded to True at the orchestrator level. The agent runs against real production data, reads real deals, generates real plans, and writes nothing. The ops lead reads a would-have-done log for three to five days. We catch drift. We tighten the prompt. We add an example to the few-shot bank. Then we flip the flag, one tool at a time.

The dry-run period is the most valuable testing window we have. It surfaces the cases nobody mentioned in scoping: the deal that has been in Negotiation for two years because the client likes it there, the prospect whose first name is just "Ahmed" because the CRM was imported badly, the test record from 2019 that still has the CEO's personal email on it. No staging environment reproduces that.

A kill switch the ops lead can actually hit

A kill switch that lives in a config file you have to SSH into is not a kill switch. It is a postmortem with extra steps.

The real one is a button. We expose it in a tiny dashboard the ops lead bookmarks on day one. One click sets a flag in Redis. The agent checks that flag at the top of every loop iteration. If the flag is set, the agent returns immediately, no tool calls, no model round-trip.

def should_halt(scope: str = "global") -> bool:
    return redis.get(f"agent:halted:{scope}") == b"1"

def step(state):
    if should_halt() or should_halt(state.customer_id):
        return state.with_status("halted")
    # ... otherwise plan, call tool, loop

Why Redis specifically? Because we do not want the halt check to depend on the same code path that is misbehaving. The check is three lines in its own file. If the agent's reasoning loop is on fire, that file still works. The dashboard that flips the flag is a single HTML page with one button. It loads in 200ms on a phone, which matters when the ops lead is on a train.

The halt-one-customer variant is worth the extra fifteen minutes to build. Same pattern, narrower key: agent:halted:customer:1234. Useful when one account is producing odd outputs and the rest of the book is fine. Without it the ops lead has a binary choice between everything on and everything off, and she will choose off and call you.

Budget caps, because spend runs away too

Containment is not only about safety. It is also about cost. An unsupervised agent loop, running against a CRM with a few thousand records, can produce a four-figure inference bill in an afternoon if a webhook misfires and the loop reprocesses the same payload a few thousand times. We have seen it happen on someone else's stack. We do not plan to see it on ours.

Three caps, all enforced before the model is called:

LIMITS = {
    "tokens_per_run": 40_000,
    "tool_calls_per_run": 30,
    "usd_per_day": 25.00,
}

When any limit trips, the agent halts the same way it would for the manual kill switch. Same file. Same Redis key. The ops lead gets a Slack ping with the limit that fired and the last tool call that ran. No surprise on the invoice. No need for someone to be watching a Grafana panel at 4am to catch it.

Reversibility, where you can have it

Some tool calls are reversible by their nature. Adding a note is reversible. Moving a deal stage is reversible. Sending an email is not. Triggering a Stripe charge is not. Deleting a record is not, in the timeline the ops lead cares about.

So we tier tools by reversibility, and the agent's permission to call them follows the tier. Reversible tools the agent calls freely. Semi-reversible tools (state changes, task creation) it calls inside the dry-run-then-flip workflow above. Irreversible tools it does not call at all, ever. It drafts the email. A human sends it. That rule has not cost us a single hour of automated work and has saved us several long conversations.

The audit log nobody reads, until they do

Every tool call writes a row. Actor, verb, target id, before-state, after-state, prompt-hash, model version, timestamp. Plain Postgres table, indexed on actor and on target id. Nothing fancy.

For six months the table is a curiosity. Then someone in finance asks why a particular deal got moved to "Closed Lost" in March, and the answer is in the table in two seconds, with the exact prompt that produced it. The audit log is what turns "the AI did it" into "here is exactly what happened, when, and why."

It is also what makes a GDPR data-subject request answerable. The agent is a processor, not a controller. The audit log proves it. We have had two of those requests on agent-touched data so far; both took under an hour to satisfy because the table was already there.

What this costs you

None of this is hard. It is maybe two days of work on a one-week build. The cost is in restraint: you build the smallest tool, not the most flexible one. You ship dry-run first and resist the urge to flip writes on for the demo. You put the kill switch in front of the ops lead before you put the agent in front of them.

The payoff is that the agent gets to stay live. Almost every agent we have killed in production was killed because containment was an afterthought and somebody got nervous. Almost none of the ones we contained properly have been turned off.

When we built the HubSpot triage agent for a SaaS client this spring, the thing we ran into was that their pipeline definitions changed mid-quarter and our set_stage tool started refusing newly-valid stages. We ended up making the pipeline a hot-reloadable config the ops lead owns, not something baked into our deploy, so she can add a stage at 9am without paging us. That kind of small surrender of control is what we keep finding in AI agent work: the operator who lives with the system needs more levers than you expect.

If you have an agent live against a system of record today, the smallest useful thing you can do this week is add a dry-run flag to one write tool and shadow it for three days. You will find at least one surprise. The whole containment story starts there.