Let Pilot continue when the next step's control is on the page

[DavertMik]

Problem

Session PrimitiveWideGold610 (Langfuse trace 731f1aa67d1e6d4e8991d90f54027541): the scenario was "Launch the currently displayed plan and verify that the launch workflow produces a visible execution or run context." Tester clicked Launch, landed in the "New Manual Run" dialog, asserted only that the plan title was visible there, and called finish.

Pilot read the state correctly — its own visual analysis said "The Launch button is visible and enabled, but it has not been activated; therefore, no resulting execution/run context is visible yet" — and returned {"decision":"fail","guidance":null}. The run ended at the open dialog, one click short of its goal.

The recovery channel exists (continue + guidance → testerConversation.addUserText) and was unreachable. reviewDecision's prompt made the two verdicts indistinguishable for an incomplete flow:

• "fail": scenario was attempted but the goal was not achieved.
• "continue": tester hasn't completed the goal; provide concrete guidance.

and then three separate clauses narrowed continue to an evidence gap — while this was an action gap. A 3:1 framing majority sends every incomplete-but-reachable state to fail.

Change

One decision site, src/ai/pilot.ts, prompt only.

• fail and continue become exact complements: fail when no further step toward the goal is available on the current page; continue when the control for the NEXT step is present there (guidance must name it), or a concrete missing check would change the verdict. "Next step" excludes retrying the step already taken, so a tester that merely repeats itself still lands on fail.
• The duplicated decision list in the user message is deleted — it restated the system prompt's DECISION block with the narrower continue wording, and its last line duplicated buildSharedEvidenceRules verbatim.
• The evidence-only clause drops out of the system prompt opener.

The wording stays general: dialogs, confirmations, wizard steps and multi-step forms all fit "the control for the next step is present."

Trade-off

More continue verdicts, so a tester stopping mid-flow now costs iterations before a verdict. The narrowing being removed arrived inside a broad feature commit (15bda39), not a dedicated anti-loop fix, so no recorded loop depends on it. Remaining brakes: MAX_ITERATIONS = 30, shouldStopForStalledExecution (forces final review after 3 iterations with no state change and no successful action — note a successful assertion resets that counter), and Pilot seeing its own prior Pilot: continue — notes in <notes> on the next review.

Testing

• bun test tests/integration/ — 149 pass, 0 fail
• bun test tests/unit/ — 1415 pass, 0 fail
• bun run format — clean

Behavioral change is in a prompt, so it wants a regression run to confirm: expectation on a replay of this scenario is one continue verdict whose guidance names the remaining step, then 1-2 more tester actions and a pass — and zero fail verdicts on a state where the next control is enabled.

🤖 Generated with Claude Code