Stop reading a passed step's result as its error

[DavertMik]

Found while investigating a failed Tester session (Langfuse trace 078285ca5b32d556be2695c50141c5c8, session VariableIncreasedBrown198). Every successful form call in that run reported a contradiction:

{
  "success": true,
  "message": "Form completed successfully with 4 commands.",
  "commandsExecuted": 4,
  "attempts": [{"command": "I.pressKey([\"Meta\",\"a\"])", "success": false, "error": "[object Promise]"}]
}

The bug

attachStepLogger binds one listener to both step.passed and step.failed, and reads argument 2 as the error either way:

const executed: ExecutedStep = { command: step.toCode(), success: !error };
if (error) {
  executed.error = errorToString(error);
  batchFailed = true;
}

But CodeceptJS emits the passing step's own return value as that argument — codeceptjs/lib/step/record.js:57:

event.emit(event.step.passed, step, val)   // val = step.run(...args)

val is a promise, so errorToString falls through to error.toString() and yields "[object Promise]".

The phantom entry is the visible half. The damage is the other half: it sets batchFailed = true, and if (batchFailed) return then drops every later step. That is why only one attempt appears where four commands ran, and why assertionsTarget keeps only the first assertion of a multi-line block — the same array that feeds recordVerification and the generated test file.

Pass/fail is unaffected: attempt() keys off whether execute() throws. What changes is what gets reported — which is what the AI reads before choosing its next action.

The fix

Pass only the step for step.passed.

Tests

The existing tests in tests/unit/executed-steps.test.ts all emit step.passed with a single argument, which is exactly why this stayed invisible. The new test emits the result alongside the step, as CodeceptJS does, and asserts all three passing steps are recorded. On main it records 1 of 3.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JCYmFZvUsENZBx71Yqojs6