Judge a blocked action on the block, not on the guard the plan named

[DavertMik]

Session SweetRequiredViolet966 (trace 1a2ef014d2e2f5d74b731507132e8515) reported a failure for a test the app passed. Scenario: "Attempt to save a manual plan without entering a title and verify that creation is prevented by validation feedback." Creation was prevented — Save stays disabled while Title is empty, the form stayed open, no plan was created.

Three things came out of reading it.

Pilot failed the test because the guard was the wrong shape

The plan predicted a validation message; the app blocks by disabling the control. Pilot's progress review acknowledged the block worked and then said:

record the scenario as failed because missing-title feedback cannot be triggered while Save is disabled

Tester obeyed, record(status:"fail") settled that expectation FAILED, hasAchievedAll() went false, and finishTest (src/ai/tester.ts:709) wrote FAILED.

Pilot's "Already-achieved detection" paragraph only describes goals met by something happening. A negative scenario's goal is met by something not happening, so Pilot fell back to the planned expectation's literal label. One line added to that paragraph:

A goal of "action must not succeed" is met by any guard that blocks it — the predicted guard is a route, not a requirement.

Same principle as #213, one decision site over — that one grounds capability presence, this one grounds the verdict.

Navigator could not assert a URL

Both URL checks in the session failed while the page sat on exactly that URL:

• verify("URL contains \"/plans/new/manual\"") → failed
• verify("Current URL is /projects/…/plans/new/manual") → failed, one call after context() returned that same URL

rules/navigator/verification-actions.md is a closed vocabulary — it ends with [DO NEVER USE OTHER CODECEPTJS COMMANDS THAN PROPOSED HERE] — and listed no URL assertion, so Navigator reasoned "There's no I.seeInURL", "I think CodeceptJS has I.seeInCurrentUrl? Not standard", then emitted a weaker I.seeInSource that failed. It is standard: seeInCurrentUrl/dontSeeInCurrentUrl exist in the Playwright helper and are already accepted by ASSERTION_STEP_NAMES (src/action.ts:557) and src/playwright-recorder.ts:328. Only the vocabulary omitted them, so they are added.

This did not cause the failure — Tester routed around it ("we could skip verify… but we can't prove") — but every URL claim was unverifiable.

Pilot's completion review was unreachable

record calls pilot.reviewCompletion() under task.isComplete() (src/ai/tester.ts:1129), but reviewDecision opens with if (task.hasFinished) return false (src/ai/pilot.ts:99) and hasFinished is status === DONE || this.isComplete() (src/test-plan.ts:282). The trigger condition was also the bail-out condition, so the call returned immediately every time — the trace has no verdict observation and no Pilot call after the last guidance.

Removed. Zero behavior change: Pilot still never reviews a records-completed test. Making it actually run means changing hasFinished, which also feeds isSuccessful/hasFailed/plan.isComplete — worth doing, but not here. The no-Pilot branch is kept as-is because unit tests construct pilot-less Testers and it uses a looser pass criterion than finishTest.

Testing

bun test tests/integration/ tests/unit/ — 1564 pass, 0 fail. Format and lint clean.

Replay expectation for the same scenario: no fail guidance from Pilot, 3/3 expectations settled passed, hasAchievedAll() true → PASSED.

🤖 Generated with Claude Code

[DenysKuchma]

With Pilot enabled, recording the last expected outcome makes task.hasFinished true via isComplete(), but leaves task.result null. The loop then exits before finalReview() because of the if (task.hasFinished) break guard. Please keep reviewCompletion() here, or separate “all expectations settled” from the terminal state, otherwise these tests finish without a Pilot verdict