AI Agent Dry Runs: Rehearsing Delegated Work Before It Acts

How to use dry runs, simulations, previews, and rehearsal environments so AI agents can prove their path before touching real systems.

A supervisor arranges a simulated AI agent workflow with blank colored blocks, folders, indicator lights, and a boundary gate.
On this page

An AI agent that can act should also be able to rehearse. The rehearsal is where the system learns the shape of the work without yet spending the authority that makes the work consequential. It lets the agent discover missing context, wrong assumptions, brittle tool calls, confusing approvals, and bad stop conditions while the cost of being wrong is still low.

Dry runs matter because delegated work often hides risk inside ordinary steps. A customer reply can look harmless until it selects the wrong account. A code change can look small until it touches a shared migration. A research summary can look polished until it relies on a stale source. A record update can look routine until the agent applies it to every duplicate match. The dry run gives the workflow a chance to show its path before that path becomes real.

This guide sits between AI Agent Sandboxes and AI Agent Evaluations . A sandbox gives the agent a safer place to work. Evaluations test behavior across a suite of cases. A dry run rehearses a particular task and returns a preview for a person or system to inspect before an action is allowed. A simulation may instead use synthetic inputs or a mock environment to explore possible behavior. Both are useful, but neither guarantees that the live system will behave exactly as rehearsed.

Choose the rehearsal that fits the question

ApproachUseful whenWhat it does not establish
Plan or draftYou need an early review of an intended approach or message.Whether the real tools can carry it out or which records they will affect.
Preview against the targetThe tool can safely calculate the actual scope and proposed changes.That the target will remain unchanged or that the later action will succeed.
Simulation with fixturesYou want to exercise ambiguous matches, failures, or other repeatable scenarios without using live records.That the fixtures capture every real condition or that a live integration works.

These approaches can be combined. For a consequential record update, a team might test failure cases with fixtures, then produce a fresh preview against the actual record for approval. Label each step as executed, simulated, or skipped, and name the environment and data used. A simulated success is evidence about that scenario, not proof of the live outcome.

A dry run is not a weaker run

The most common mistake is treating a dry run as a polite summary. The agent says what it plans to do, the person nods, and the real action proceeds almost unchanged. That is useful only when the task is simple enough that the plan itself reveals the risk. A useful dry run exercises as much of the real path as is safe, while preventing the actions that would change the target system. Where that is impossible, it should say which steps were simulated or skipped.

If the agent would search a knowledge base, the dry run should search it. If it would match a customer record, the dry run should perform the match and explain the confidence. If it would update files, it should prepare the diff. If it would send a message, it should compose the exact message, recipient, identity, and attachments. If it would run a tool that changes state, the dry run should call a preview version of that tool or produce the same structured payload the real tool would receive.

That distinction keeps rehearsal honest. A vague plan can hide problems behind intention. A dry run with tool-shaped outputs exposes them. The reviewer can see that the account match was weak, the file diff reached outside the assigned path, the source was older than the policy, or the agent is about to use a permission it was not meant to use. The dry run makes the intended path inspectable, but it cannot prove that every later tool call or external system will behave the same way.

Preview the consequence, not only the content

Many agent workflows preview the visible artifact but not the action around it. A draft email is shown, but the recipient is hidden. A code patch is shown, but the commands that will run after commit are not. A database update is summarized, but the affected record set is not. That leaves the reviewer judging prose while the risk lives in scope.

A useful dry run names the consequence. It should say what would be created, edited, sent, deleted, published, charged, archived, retried, or escalated. It should also name what would remain untouched. A preview that says “three records would be updated” is weaker than one that shows the three records, the fields, the before state, the proposed after state, and the reason each record is in scope.

This connects directly to AI Agent Permissions . Permission ladders are easier to trust when the rung change is visible. The dry run is a natural place to ask whether the agent is still reading, preparing, requesting approval, or acting. If the agent is only approved to draft, the rehearsal should stop before sending. If it is approved to prepare a refund, the rehearsal should stop before issuing it. The dry run turns authority into something concrete enough to inspect.

Set the boundary before starting

A dry-run label is not a safety control by itself. Inventory every tool the agent can reach and identify side effects beyond the final send or update: creating drafts, writing files or logs, reserving inventory, triggering webhooks, consuming paid API calls, and exposing data to a service. Even a lookup can disclose information or incur a charge. Decide which effects are allowed, which must be redirected to a test environment, and which must be blocked.

Enforce that boundary outside the agent’s instructions: use read-only credentials where possible, disable or intercept write-capable tools, and test the controls with a harmless attempted write. Confirm what each tool’s preview mode actually promises; a wrapper that merely hides the last button may still allow earlier mutations. Kubernetes, for example, requires admission webhooks with side effects to suppress them for dry-run requests; the Kubernetes guidance on webhooks illustrates why downstream behavior matters. If a necessary step has no safe preview, stop before it and label the missing evidence rather than claiming the whole workflow was tested.

A test environment needs its own boundary: disable production credentials, outbound email, payment endpoints, and webhooks unless a specific safe substitute is configured. Check where the tool actually points before rehearsal. If a preview endpoint is unavailable, do not silently fall back to the live write endpoint; stop and report the gap.

Live data can still make a rehearsal sensitive. Use the least data needed, prefer synthetic records for repeatable tests, and restrict access to previews and traces. AI Agent Test Fixtures and AI Agent Data Boundaries cover those choices.

Simulation should preserve friction

A dry run can become misleading when the simulated world is too clean. If the test records are perfectly named, the tools always respond, the sources never conflict, and approvals are obvious, the agent learns to succeed in a polished miniature rather than in the operating environment. Rehearsal should preserve the friction the real task is likely to contain.

That does not mean every dry run needs chaos. It means the preview should keep the parts that matter. Duplicate records should remain duplicate. Missing fields should remain missing. Tool latency should be represented enough that the workflow does not assume instant response. Conflicting sources should be visible rather than resolved by convenience. If the agent will encounter stale knowledge, partial access, dirty worktrees, or ambiguous customer language, the dry run should not quietly remove those conditions.

AI Agent Knowledge Bases makes this especially important. A grounded answer depends on source selection. A rehearsal that uses a friendly sample document may prove little about how the agent will behave when the real knowledge base contains overlapping policies and older drafts. A better dry run asks the agent to show which source it would use, which sources it considered, and why the chosen source governs the task.

A single successful rehearsal samples one path. Model responses, search results, timing, and external state can differ between runs. For a recurring workflow, run representative cases more than once, including cases that should stop, and inspect failures as well as successes. Record the tool and model configuration where practical. Repetition can expose variation, but it cannot establish a failure rate without a representative test set and enough observations.

Design tools with preview modes

Dry runs are much easier when tools are built for them. A tool that can only act or fail forces the agent to simulate in prose. A tool that supports preview, validation, and commit lets the workflow separate judgment from consequence. The preview call can return the matched records, the proposed changes, warnings, required approvals, estimated cost, and a stable token that the commit call later uses.

That pattern is a close cousin of AI Agent Tool Contracts . A good contract does not simply expose power. It exposes the shape of the action before the action occurs. The preview should be boring, structured, and inspectable. It should avoid clever natural-language summaries when a table-shaped or diff-shaped result would be safer. It should make the agent’s next step obvious enough that a reviewer does not need to guess.

Preview modes also reduce accidental drift between rehearsal and execution. If the dry run generates one payload and the live action recomputes everything from scratch, the agent may pass the preview and then act on changed inputs. For consequential work, the live step should either reuse the reviewed payload or clearly mark what changed since review. A dry run that cannot be tied to the eventual action is closer to advice than control.

A stable token is useful only if the service enforces its meaning. Bind approval to the target environment, identity, exact payload, relevant record versions, policy version, and an expiry. At commit time, check those preconditions atomically where the system supports it. If the target or payload has changed, the token has expired, or the service cannot verify the preconditions, stop and produce a fresh preview for review. A hash can identify the reviewed payload, but a hash alone neither grants authority nor prevents the target from changing.

This resembles the plan-and-apply workflow described in Terraform’s plan documentation : an earlier speculative plan can differ from the final result when the target changes, so the final plan needs review. A saved plan is useful evidence, not a promise that every external condition will stay fixed or that a later apply cannot fail.

A worked example: updating a customer record

Suppose the assignment is to change a customer’s mailing address after a verified request. The agent searches using the authorized identifier and finds two similarly named records. The preview shows both candidates, the matching evidence, the proposed address, and the source of the request. It does not select or update either record while the match is ambiguous.

After an authorized person resolves the match, the agent produces a new preview with the selected record ID and version, current address, and exact field update. The reviewer approves that specific change, not a general permission to edit customer records. The commit service checks the approval, identity, expiry, payload, and record version before applying the update. If the address changed in the meantime, it rejects the stale preview and requires another review. A successful rehearsal still cannot guarantee that the live service will be available or that a real request will contain no new ambiguity.

A compact review handoff for the resolved case might look like this:

  • Assignment and environment: update the mailing address in the production customer system, following the verified request.
  • Target and change: record ID 482, version 7; replace the current mailing address with the exact address shown in the protected preview. No other fields or records are in scope.
  • Evidence and checks: request verified; record identity confirmed by the authorized reviewer; address format validated. The preview was generated at a stated time using the current record version.
  • Decision: approve this specific update before a stated expiry, or reject it. The write has not yet occurred. If the version or proposed address changes, produce and review a new preview.

The record ID and version are illustrative; the actual preview should show the relevant values to an authorized reviewer while protecting customer data. If the service cannot enforce the version check, the workflow needs another reliable concurrency control or a human-led procedure; a manual check followed by an unguarded write still leaves a race window.

If the commit times out, do not assume it failed and immediately repeat it. Check the recorded operation or the target state first; use an idempotency key when the service supports one, following that service’s retry rules. If the outcome remains unknown, stop and escalate. The retries and idempotency guide explains why a retry can otherwise duplicate an action.

Decide what blocks promotion

Not every dry run should be allowed to proceed. The workflow needs block conditions that are known before the agent starts. A missing source, a low-confidence record match, a new permission request, an unusually large change set, an unresolved tool warning, or a validation failure should stop the run until a person or a stronger policy resolves it.

The important part is that block conditions should not be invented after the reviewer feels uneasy. They should be part of the task contract. AI Agent Acceptance Criteria explains this at the level of done. Dry runs use the same habit earlier. They define what must be true before a preview can become action.

This is also where AI Agent Output Verification becomes more useful. Verification after action can catch errors, but some errors are cheaper to catch before action. A dry run can verify that the draft cites the governing source, the diff stays inside the assigned files, the message uses the right identity, and the record update affects the expected scope. The agent should not earn promotion merely by sounding confident. It should earn promotion by showing the evidence the workflow asked for.

Rehearsal is part of the user experience

Dry runs are not only backend controls. The person supervising the agent experiences them through the control surface. If the preview is noisy, vague, or hard to compare with the final action, it will be skipped under pressure. If it is concise and concrete, it becomes a natural part of delegation.

The preview should show the original assignment, the proposed action, the evidence used, the validations performed, the warnings raised, and the exact approval being requested. It should separate what the agent knows from what it inferred. It should make remaining uncertainty visible without turning every ordinary caveat into an alarm. AI Agent Control Surfaces covers that interface layer, but dry runs give the surface one of its most useful moments: the pause before consequence.

A well-designed rehearsal also teaches the human where trust is deserved. The reviewer sees how the agent behaves when asked to prove its work. Over time, reviewed results from representative cases can inform a decision about lighter review, subject to the workflow’s policy and continued monitoring. Repeated warnings can reveal that a task is not ready for automation. The dry run becomes evidence about the workflow, not only a preview of a single action.

The rehearsal should leave a trace

Dry runs are most valuable when they become part of the run history. If the live action later fails, the team should be able to compare the rehearsal with what actually happened. Did the reviewed payload change? Did a tool return a different result? Did a human approve a warning without understanding it? Did the agent ignore a block condition? Those questions are answerable only if the preview was preserved.

AI Agent Observability gives that history a place to live. The dry run should be recorded as a meaningful event, not as a loose chat message. It should include the preview inputs, outputs, warnings, approvals, and promotion decision, with sensitive values redacted or access controlled according to the retention policy. If the agent later acts, the trace should link the action to the reviewed preview and the separate approval or policy decision that authorized it.

This is the quiet discipline behind reliable agents. The system does not ask people to trust a future action because the agent sounds capable. It asks the agent to rehearse, expose consequence, preserve evidence, and earn the next step. For low-risk work, that rehearsal may be small. For high-risk work, it may be the difference between useful delegation and an expensive surprise.

Dry runs do not make agents harmless. They make risk visible at the moment when visibility still matters. That is enough to change the operating rhythm. Instead of discovering the agent’s judgment after the action, the team can inspect it before the boundary is crossed.

Written By

JJ Ben-Joseph

Founder and CEO ยท TensorSpace

Founder and CEO of TensorSpace and the editor of Fondsites: an engineer working across software, AI, cybersecurity, and biosecurity who builds the site's games and tools.

Keep reading