Skip to content

fix(cli): a refused probe is the last word on algorithms run (#1106) - #1114

Open
HugoXK wants to merge 1 commit into
mainfrom
agent/1106-algorithms-run-hint
Open

HugoXK wants to merge 1 commit into
mainfrom
agent/1106-algorithms-run-hint

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator

Closes #1106

What changed

algorithms run ... --probe printed the plan (which ends with next:) and then the #707 quick-trial hint, because the hint's guard checked only --rounds / --workers and agent_calls_upper_bound > 50, never whether the probe refused. With gepa's defaults the plan is 292 calls, so the default invocation was the broken one; the issue's own regression row passes --rounds 1 --workers 1, which is exactly the row that hides it.

  • agentdescent/cli.py — in _cmd_algorithms_run, the hint prints only when the probe did not refuse (probe is None or probe.get("ok", True)); with a refusal the run has nothing to start. Every other line is byte-identical: the then:/next: shape, the probe block's repair line, and the hint's order after a passing probe or with no probe are untouched.
  • tests/test_plan_probe.py — one red row beside test_algorithms_run_human_form_exits_nonzero_too on the default shape: exit 1, the last stdout line starts with next: renew the subscription, and (for a quick trial appears nowhere. Plus two guard rows: the hint still prints, last, with no probe and after a passing probe (both already pass at base).
  • changelog.d/1106-algorithms-run-hint.md — one Fixed fragment.

No docs edit: docs/algorithms.md:26-28 becomes true as written.

Red at live main 5048349a

Clean worktree at 5048349aaceb1ad4c93318581f9a2b49481a5a99, with only tests/test_plan_probe.py from this branch copied in:

FAILED tests/test_plan_probe.py::test_a_refused_probe_on_the_default_shape_ends_on_the_next_line
E       AssertionError: ['then: agentdescent plan /tmp/pytest-of-xuke/pytest-0/test_a_refused_probe_on_the_de0/algo.spec.json --probe', 'next: renew the subscription at the provider, or point api_key_env / model at another plan', '      (for a quick trial: re-run with --rounds 1 --workers 1)']
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7f4275438030>('next: renew the subscription')
E        +    where <built-in method startswith of str object at 0x7f4275438030> = '      (for a quick trial: re-run with --rounds 1 --workers 1)'.startswith
1 failed, 2 passed, 13 deselected

The two passing rows are the guards above: they pass at base and at the tip, so the fix cannot pass by deleting the #707 hint.

The issue's table, measured at base and at the tip

Offline endpoint answering the measured Ark expired-subscription body (HTTP 400), gepa, fixture 12 cases; exit and the last human stdout lines, no key, no network:

row base 5048349a tip 64db363d
A defaults + --probe, refused exit 1, last line (for a quick trial: re-run with --rounds 1 --workers 1), next: second-to-last exit 1, last line next: renew the subscription at the provider, or point api_key_env / model at another plan, hint absent
B --rounds 1 --workers 1 + --probe, refused exit 1, next: last, hint absent unchanged
C defaults, no --probe (control) exit 0, hint last exit 0, hint last

Row C is the one that must not regress: a run nobody refused still names the quick trial.

Verification at the tip

Tip SHA: 64db363

Ran, at this tip:

  • pytest -o addopts="" -q tests/test_plan_probe.py tests/test_repo_hygiene.py tests/test_cli_rounds_workers.py → 27 passed
  • python -m tools.gen_api_docs --check → docs/api.md is up to date
  • python -m tools.changelog_fragments check → 50 changelog fragment(s) OK

Not run:

  • full pytest -q — the host-only legs (docker sandbox, test_era_srbench, test_genesis_example, wheel smoke) fail in this box at base too, and none is in a file this PR touches; CI's test (3.9)–(3.13) is the line
  • mkdocs build --strict — no docs/ or mkdocs.yml change

Branch agent/1106-algorithms-run-hint was rebased onto 5048349a with a --force-with-lease pinned to a9c7ef06; the patch is byte-identical to the pre-rebase one (git diff f5cc52ef a9c7ef0 and git diff 5048349a 64db363d have the same sha256, 30bf58ec…).

Size: 3 files, +65/−2 against base 5048349a. T2 by .github/agent/risk-rules.json (agentdescent/**); the risk:T* label is the controller's.

maintainer_can_modify: true is asked for in the create call. This PR's head is a same-repo branch, and GitHub only tracks that flag for fork heads — a same-repo head reports it null/false structurally, so the field is recorded as requested rather than re-read as a boolean.

Written-by: AgentDescent Dev-2
Bot: AgentDescent Dev-2

Claim-Id: #1106

@HugoXK HugoXK added bug Something isn't working area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) agent-pr Pull request opened by an agent labels Nov 16, 2026
With the kind's default rounds and workers the plan is large enough for the
#707 quick-trial hint, and a refused `--probe` printed the hint after `next:`,
so the last human line answered a question the refusal had already answered.

The hint now prints only when the probe did not refuse (`probe is None or
probe.get("ok", True)`). Nothing else moves: `then:`/`next:`, the probe block's
repair line, and the hint order after a passing probe or with no probe stay
byte-identical, and the two passing rows in `test_plan_probe.py` pin that.
@HugoXK

HugoXK commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed tip: 64db363d89b1eb2f6bc97ea4ee7fadcb7787cd0c — ready to judge at this sha.

  • build, test (3.9)–test (3.13), otel-wire all success, deploy skipped, both report-only checks success. mergeable_state: clean.
  • Runner failure on the record, stated plainly: the first run of test (3.10) on this same sha failed in tests/test_sandbox_container.py with docker: toomanyrequests plus a podman TimeoutExpired at 120s — the unauthenticated Docker Hub pull limit. The rerun on the identical sha came back success. That is a runner-side quota, not this diff: nothing in the change touches container runtime, image pulls, or the podman timeout, and no code or test was altered for it.
  • Scope, red evidence at base 5048349a and the three measured rows (A broken at base, B unchanged, C the control) are in the body above; nothing has moved since.

A review on this sha is what this comment asks for; it is not a verdict and it is not a merge.

@HugoXK HugoXK left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review tip: 64db363

Verdict: PASS — no must-fix items.

Scope: 3 files, +65/−2 — agentdescent/cli.py, tests/test_plan_probe.py, changelog.d/1106-algorithms-run-hint.md. No .github/**, no dependency or lockfile, no installer or host path, no FORWARDED_ENV/ARK_API_KEY, no new env read, no network or write.

1. The refusal path is not changed — agentdescent/cli.py:4649 still returns 0 if probe is None or probe.get("ok", True) else 1, the same contract as the JSON branch at cli.py:4605; the refused next: and the then: line are untouched. The new condition at cli.py:4641-4643 gates only the print at cli.py:4647, and it is the same expression as the exit code, so the hint cannot print while the exit code says the probe refused. It is text-only: no branch that starts work is added, removed or reordered, and algorithms run does not start a run in any case.

2. No leak — the gated line is static text (--rounds 1 --workers 1); the change adds nothing that prints from probe, and the probe block itself is unchanged.

3. The test pins behaviour, not layout — tests/test_plan_probe.py:282 requires exit 1, the last stdout line to start next: renew the subscription, and the hint to be absent; tests/test_plan_probe.py:304 and :314 keep the hint last with no probe and after a passing probe, so deleting the hint fails the suite rather than fixing the row.

Evidence at this tip: red at base 5048349a with only this branch's tests/test_plan_probe.py copied in = 1 failed, 2 passed, 13 deselected (the refused row); at 64db363d the same selection = 2 passed, and tests/test_plan_probe.py tests/test_repo_hygiene.py tests/test_cli_rounds_workers.py = 27 passed. python -m tools.gen_api_docs --check = up to date; python -m tools.changelog_fragments check = 50 fragments OK. Every check run at this head is success except deploy (skipped): the five test legs, otel-wire, build, and the two report-only jobs.

Non-blocking: the readiness expression is repeated at cli.py:4605, :4643 and :4649; a named local would keep a future probe flag in one place. Not a must-fix for this change.

This is an internal AgentDescent Security review, not the PASS that .github/agent/review-policy.json counts — that file lists only the login Birfy. It is advisory only: it is not the merge gate and it authorises no merge.

@HugoXK HugoXK added the awaiting-approval Green T2/T3 PR: waits for a PASS at its tip and Danyang's merge click label Nov 16, 2026
@HugoXK
HugoXK requested a review from Birfy November 16, 2026 07:29
@github-actions github-actions Bot added the risk:T2 Risk tier T2 (package code or an unnamed path), set by the controller from the diff label Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-pr Pull request opened by an agent area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) awaiting-approval Green T2/T3 PR: waits for a PASS at its tip and Danyang's merge click bug Something isn't working risk:T2 Risk tier T2 (package code or an unnamed path), set by the controller from the diff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): a refused probe on algorithms run still ends with the quick-trial hint

1 participant