A model may propose anything and commit nothing

The standard way to build an LLM feature that changes data is to define tools, describe them to the model, and execute whatever it calls. Every provider's documentation demonstrates this. It works, and for a surprising number of products it is fine.

It stops being fine when the data has consequences. If the record is a financial ledger, a patient file, a payroll run, or anything a regulator might later read, the question is no longer whether the model is usually right. It is what happens on the occasion it is not.

The failure you are actually designing for

A model with direct write access fails in a specific and unpleasant way: silently and plausibly. It does not throw. It produces a well-formed call with a wrong argument, the write succeeds, and nobody notices until something downstream does not add up — possibly months later, possibly during an audit.

This is worse than a crash. A crash is loud, attributable, and fixed once. A silent bad write is discovered late, is hard to trace back to a cause, and by the time you find it there may be many of them.

The pattern

Split proposal from execution. The model never mutates anything. It produces a structured, described, reviewable intent that is stored as a pending action. A separate step — a human, or in some cases a deterministic validator — turns that intent into a real change.

model turn
  └─ emits: intent { operation, arguments, rationale }
        │
        ▼
  validate shape, permissions, preconditions
        │
        ▼
  persist as pending action  ──────►  surfaced for review
                                            │
                                    explicit confirmation
                                            │
                                            ▼
                                      execute + audit

Four things change once you do this.

• A wrong model output becomes a rejected suggestion instead of a corrupted record. The blast radius of a bad generation drops to zero.
• You get an audit trail for free. Every change has a stored intent, a rationale, and a confirming actor attached to it.
• Rejections become training data about your own system. A pending action that users consistently decline is telling you something specific about where the model is weak.
• You can raise model temperature, swap providers, or try a smaller model without the risk calculus changing, because the safety property lives in the architecture rather than in the model's behaviour.

What it costs

Friction, and it is real. A conversation that could have been one turn becomes two. Users who wanted the system to just do the thing have to confirm that they wanted the thing done. On operations where the cost of being wrong is genuinely low, this is a bad trade and you should not make it.

So the design question is not whether to use the pattern, but where the line sits. I have found the useful test to be: if this operation were wrong, would we find out quickly, and could we reverse it cheaply? Two yeses mean direct execution is fine. A single no means it belongs behind confirmation.

Note that this is a question about the operation, not about the model. It is tempting to gate on model confidence instead — execute when confident, confirm when uncertain. Do not. Self-reported confidence is poorly calibrated for exactly the cases that matter, and a model that is confidently wrong will route itself straight past the safeguard you built.

Reviewability is a design constraint

A pending action only helps if a human can actually judge it. An intent rendered as a raw payload is not reviewable — people approve it without reading, and you have added friction while removing no risk.

That means the intent has to be described in the user's terms, not the system's. Not the table and column that will change, but what will be true afterwards that is not true now. If a reviewer cannot tell a good proposal from a bad one in a few seconds, the confirmation step is theatre.

Why this generalises

The same shape shows up well outside the model-writes-to-database case. Coding agents that open a pull request rather than pushing to main are doing this. Infrastructure tools that print a plan before applying it have done it for a decade. Deployment pipelines with a manual promotion gate are doing it.

None of these are AI patterns. They are the same old idea — separate what is proposed from what is committed — applied to a component that happens to be probabilistic.

Which is the part worth holding on to. Building reliable systems on unreliable components is not a new problem, and the accumulated answers still apply. The novelty of the component is not a reason to abandon them.