Skip to content

fix(cli): a failed plan --probe must not recommend evolve (#1093) - #1103

Merged
Birfy merged 1 commit into
mainfrom
fix/1093-probe-failed-next
Nov 15, 2026
Merged

Birfy merged 1 commit into
mainfrom
fix/1093-probe-failed-next

Conversation

@Birfy

@Birfy Birfy commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator

Closes #1093

--probe already exits non-zero when the model/key/subscription is refused, but the last next: line (and algorithms run --json's next) still recommended agentdescent evolve. A reader or a host that follows next then starts a run that cannot work and leaves a failed entry in the run store — the case #1048 was meant to catch.

  • When the probe refuses, the last human line is next: with the first failed row's repair action (for example, renew the subscription). A then: agentdescent plan <spec> --probe line sits just above it so the probe can be re-run after the fix. The output no longer contains next: agentdescent evolve.
  • --json keeps ok about the plan/spec itself (the plan is still valid). It adds ready: false and sets next to the same repair action. Hosts should gate on ready or the exit code, not on ok alone.
  • A successful probe, or a plan / algorithms run without --probe, is unchanged: the last line is still next: agentdescent evolve …, exit 0, and JSON is the same object as before plus ready: true when --probe ran and passed. Without --probe there is no ready or next on plan --json.

Tests in tests/test_plan_probe.py cover both verbs, human and JSON forms, the success path, and the no-flag path. The new assertions fail on main 914c626 and pass at this tip.

When --probe refuses, the last next: line and the JSON next field are
the repair action, not evolve. JSON also sets ready: false while ok
stays about the plan itself. A successful or omitted probe is unchanged.
@Birfy

Birfy commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

Self-review (Birfy)

Tip SHA: b7257c9
Ran:

  • pytest -q → 7502 passed, 64 skipped at this tip. Four failures also fail on unmodified main 914c626 and are environment: test_integrations.py / test_mcp_http_transport.py need python -P -m agentdescent.cli from the shared venv, whose editable install is missing; tests/test_plugin_host_cli.py::test_present_host_passes refuses a bare opencode agent because ARK_API_KEY is set on this box (same as on fix(eval): name the real cause of a noise refusal and stop faking gain_over_noise (#1089) #1090).
  • The new/changed tests in tests/test_plan_probe.py fail on main 914c626 (5 failed, 8 passed) and pass at this tip (13 passed).
  • python -m tools.gen_api_docs --check → pass
  • python -m tools.changelog_fragments check → pass
  • python -m tools.gen_algorithms_docs --check → pass
  • .github/scripts/risk_gate.py --pr 1103 → T2
    Not run:
  • mkdocs build --strict — docs/plugin-quickstart.md and generated docs/algorithms.md changed; the CI build job is the line
  • CI test matrix 3.9–3.13 — queued at this tip
  • Ark real-model probe — this change only rewrites the failed-probe next/ready surface; the stub tests cover the refusal path
    Findings:
  • none
    Verdict: PASS

@Birfy
Birfy enabled auto-merge (squash) November 15, 2026 22:45
@Birfy
Birfy merged commit f5cc52e into main Nov 15, 2026
9 checks passed
@Birfy
Birfy deleted the fix/1093-probe-failed-next branch November 15, 2026 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plan/algorithms run --probe: a failed probe still ends with 'next: agentdescent evolve' (and --json says ok: true)

1 participant