Test before it goes live

Ask your coding agent#

Run the test parameters for the invoice exceptions agent's script and its invoice comparison function, then write behavior tests for a matching invoice, a mismatched one and an unknown supplier. Run each test ten times and tell me which version they ran against.

Test parameters#

A script or a function keeps stored sets of inputs, its test parameters, and each set runs the whole path once. Give each path its own set, including one awkward case. A test run that writes needs an admin's go-ahead for its write list: your coding agent lists what the run will read and write, and waits.

Activating a version that puts new script text live is refused while the script calls an operation with no successful call since its last lapse, and publishing a function is gated the same way. Running the test parameters gives each operation that call. An admin can override per operation with Go live untested, recorded with their name.

Behavior tests#

A behavior test asks the agent a question many times and checks each answer against criteria, with a pass threshold. Every write is refused before it leaves, so a behavior test never changes another system. Behavior tests aren't a gate: activating a version doesn't run them, so run them before you decide.

Keep the tests in your repo with the agent's files, and accept the ones every run should check.

What you decide#

• The go-ahead for each test run that writes.
• Which behavior tests a new version must pass before someone activates it.

Done when#

• Every set of test parameters reads Verified.
• The accepted behavior tests pass on the staged version, and a person has activated it.

Test behavior covers comparing a change, such as a cheaper model, before you make it.