Skip to main content
Simulations let you ask “what would this agent do if…” before you ship a prompt change. A scenario describes a synthetic user — who they are, what they want — and the assertions that must be true of the resulting conversation. Running a scenario replays a system prompt against an LLM-driven user, produces a synthetic transcript, and scores it. Run the same scenarios against your live prompt and again against a candidate prompt, and the difference in pass rate tells you whether the edit is safe to ship.
Simulations never start a real conversation and never touch your voice provider. They test the prompt, not the deployment, so they work for every provider. They also never appear in your conversations, analytics, or billing.

Scenarios

A scenario has three parts:
  • User — a persona: their role, their goal, optional behaviour traits, facts they only reveal when asked, and an optional opening line.
  • Assertions — what must be true of the conversation. Each assertion is a question the judge answers about the transcript (yes/no, a number, or text) plus the condition that makes it pass. Assertions decide pass/fail.
  • Settings — how many turns the conversation may run for and which model tier drives it.
You don’t have to write a scenario from scratch. In the scenario editor, describe what you want to test in one line (“the agent always offers a callback to angry users”) and Draft with AI generates a user and assertions from the agent’s live prompt for you to review and edit before saving.
Your agent’s enabled monitors are also scored against the synthetic transcript, but only for information — they never decide pass/fail, because monitors are tuned to real traffic and can fire surprisingly on synthetic conversations.

Export to LangWatch Scenario

When editing a saved scenario, expand Export LangWatch code to copy a TypeScript LangWatch Scenario test. The export maps the user persona, turn limit, and assertions from your current draft. It includes an adapter stub — replace that stub with the integration that invokes your agent before running the test.

Prompt targets

A run or suite can be pointed at any of an agent’s prompts:
  • the live prompt currently deployed on the agent, or
  • any version — a deployed incarnation, or a candidate prompt drafted on the agent’s Versions tab.
Because a candidate prompt can be simulation-tested before it is ever deployed, Simulations closes the loop from “draft an edit” to “know it’s safe”.

Suite runs and compare

A suite run replays every enabled scenario for an agent against one prompt. Its pass rate is the headline number. To judge a prompt change, run the suite against your live prompt, run it again against a candidate, then open Compare to see the pass-rate delta and a per-scenario breakdown of what got fixed, regressed, or stayed unchanged — alongside a diff of the two prompts.

Enabling simulations

Simulations are opt-in per organization. A Zelto admin turns the feature on from the organization’s feature-flags tab; once enabled, owners and admins see the Simulations entry in the sidebar and can author scenarios and start runs.
  • Agents — where live prompts and candidate prompts live.
  • Monitors — the monitors that are scored (informationally) on each simulated conversation.
  • Benchmarks — compare several models and prompt revisions on the same cases, including cases created over the API.