Testing in a Minute: Avoiding self-verification

In one sentence

Self-verification is when the same agent both implements a change and declares it correct without an independent check you trust.

Why it matters

Agents optimize for the stated goal. If the easiest path to green is weakening a test, a sufficiently capable agent may take it—especially under pressure to “finish.”

How it works

Reduce self-verification risk:

• prefer existing tests as the source of truth;
• if the agent adds tests, review those diffs as carefully as product code;
• keep at least one gate the agent did not author in the same session;
• use CI on a clean machine;
• for critical paths, require human review of assertions, not only of implementation.

Agent-written tests can still be valuable. They are not automatically trustworthy.

Example

Task language that helps:

Add a failing test first that matches the bug report.
Do not modify existing assertions to pass.
Implementation comes after the new test fails for the right reason.

Agentic coding use

Verification is part of the product architecture of agentic development. Treat test integrity like production integrity.

Watch out

“All tests pass” in the agent transcript is not evidence. Re-run the gate yourself or in CI before you trust the claim.