# Test behavior

> Proves every step of a script works, checks how an agent behaves across many runs, and compares a change before you make it.

**Skill for your coding agent:** [`mindset-test-agent/SKILL.md`](https://docs.mindset.ai/skills/mindset-test-agent/SKILL.md)

Running an agent is a loop: see what happened, diagnose what went wrong, change the agent, test the change, activate it, then see what happens next. Highlighted: Test.

## Ask your AI client

```
Write acceptance tests for the invoice exceptions agent: it calls the comparison function, its supplier query names the invoice and PO numbers, and it never registers a query when the invoice matches. Run them, then compare it on a cheaper model.
```

Skill: [mindset-test-agent](https://docs.mindset.ai/skills/mindset-test-agent/SKILL.md)

## What you get back

Two kinds of test, and your AI client runs both.

|                   | Test parameters                                                             | Acceptance tests                                         |
| ----------------- | --------------------------------------------------------------------------- | -------------------------------------------------------- |
| Proves            | Every step of a script or function runs, and every operation it calls works | The agent behaves as it must, across many runs           |
| Writes            | Real, within a list an admin agreed                                         | Refused before they leave                                |
| Result            | **Verified**, or each step says why not                                     | Pass or fail against a threshold, with a recorded reason |
| Blocks going live | Yes, over untested operations                                               | No. You read the result and decide                       |

The go-live gate and what makes an operation lapse are on [Versions, testing and going live](https://docs.mindset.ai/docs/versions-and-going-live).

## Test parameters

Keep one set per path through the script, so every branch is taken: an invoice that matches its PO, one with a difference the contract allows, and one it doesn't, which registers a supplier query. A step that wasn't exercised says why: **Branch not taken**, **Not reached**, **Refused**, **Write not agreed** or **Failed**. **Branch not taken** means you need another case.

> **Warning:** A test run's writes are real. Point the inputs at somewhere safe, such as a test supplier record, before you give the go-ahead.

A go-ahead covers that exact write list and is recorded with who gave it. If the list changes, you're asked again. Tested status is kept per environment, so run them in Production too.

## Acceptance tests

A test is a situation plus criteria. Some are checked by code (`invokesTool`, `doesNotInvokeTool`, `toolParametersContain`, `answerContains`, `respondsWithin`), others judged by a model (`doesNotHallucinate`, `staysInCharacter`, `responseGroundedInKnowledge`, `satisfiesAssertion`). A refused write still counts as a call, so test what the agent tried to send with `toolParametersContain`.

Results are statistical. A judged criterion runs 5 times by default and passes when the lower bound of a 95% confidence interval on its pass rate clears 0.5, which at 5 runs means 5 out of 5. You can raise the runs (up to 200) or the threshold.

A **simulated user** tests a conversation instead of one answer: a second model with a persona and a goal, such as a supplier who wants the query dropped without sending a delivery note. The whole conversation is scored.

An accepted test can't be edited or deleted. **Amend** replaces it and **Retire** stands it down, each with a reason, so the bar only moves where people can see it.

## Compare a change before you make it

An experiment runs your accepted tests against a changed setup, a different model or part of the system prompt, beside the current one. Each criterion comes back **Improved**, **Regressed** or **No detectable difference**, with cost and speed for each. Nothing touches the live agent, and there is no way to split live traffic between two versions.

## What you decide

- What the agent must always do, and at least one thing it must never do.
- Where test writes land.
- How much to spend. Tests run the agent many times, and the judge and simulated user are model calls too.
- Whether a version is good enough to activate. Failing acceptance tests don't stop it.

## Done when

- Every script and function the agent runs reads **Verified** in the environment it goes live in.
- The agent has accepted tests for what it must do and must not do, and they run daily.
- You can say which passed on the version you last activated.

## In the Studio

Test parameters run from the agent's **Script** tab, or a function's **Preview** tab, with **Run**. Acceptance tests live in **Optimize → Acceptance tests**: describe a test to the **Optimize Agent**, **Dry-run** the draft, **Accept** it, then **Run all**. Under **Scheduled runs**, **Run these tests daily** runs the set at a UTC time. **Optimize → Experiments** compares a change, and **Optimize → Runs** shows what each test run cost.

![Optimize → Acceptance tests for the Invoice exceptions agent.](https://docs.mindset.ai/images/docs/studio/optimize-acceptance-tests.png)