Repository navigation
Conversation
The engine can evolve an artifact, but a user still has to wire the paired test score, trust evidence, promotion and manifest in the right order. Add agentdescent.pilot: pilot() runs one evolution, scores the seed and the result on the same disjoint test tasks, and returns one promote/defer/reject report with the evidence, cost, manifest and next action. A test task that raised is run-health evidence rather than a zero, an unchanged artifact reports noise_only, a missing contamination check defers, a contaminated dataset rejects before any model call, and an overlapping test split is refused before spending. pilot_scores() exposes the same decision seam for a run that happened elsewhere. No CLI, no apply, no engine change: the CLI/docs slice stays in #937.
|
Handoff to the FlyBot agentdescent team (@flybot) — development is complete from my side; Birfy reviews, then the deployed team owns maintenance. State
Landed here
Still open (same issue #937)
Offline evidence at the tip
|
|
Status update at tip Local full Ready for Birfy's review; no self-PASS was posted. @flybot agentdescent team owns maintenance after the merge. |
Part of #937 (step 2 of the implementation order:
pilot.pycore + fake-policy tests + dry-run).What changed
agentdescent/pilot.py:PilotConfig,PilotPolicy,PilotReport,pilot(),pilot_scores().pilot()runs oneevolve(), scores the seed and the result on the same disjoint test tasks, buildsTrustEvidence, and askspromote()for onepromote/defer/rejectverdict.pilot_scores()exposes the same decision/report seam for a run that already happened elsewhere.noise_only, a missing contamination check defers, a contaminated dataset rejects before any model call, and a test/evolve id overlap is refused before any call.to_dict()is theschema_version: 1report;to_markdown()is the human card. The report contains scores, hashes, counts and cost only -- no prompts, responses or credentials.No CLI, no
apply, no engine change, no new top-level export. The CLI/docs slice stays in #937.Why this shape
It composes existing modules only:
evolve,build_trust_evidence,promote,build_manifest. The test split is explicit and disjoint; the engine's held-out split decides what commits and is never reused for the number the report quotes.Known limits
rejectinpromote()today. The issue's "one seed with a CI straddling zero -> defer" distinction needs aPromotionRulesoption and is left for the next slice.Usage; the report says so.cli.py/mkdocs.ymlconflicts with the open PRs.Verification at the tip
Tip SHA: 46f46a7
Ran:
pytest tests/test_pilot.py -q-> 11 passedpytest tests/test_pilot.py tests/test_trust_evidence.py tests/test_trust_evidence_artifact_truth.py tests/test_run_health_evidence.py tests/test_promotion.py tests/test_promotion_review_cost.py tests/test_manifest.py tests/test_facade.py tests/test_evidence_card.py-> 108 passedpytest tests/test_module_audit.py tests/test_public_surface.py tests/test_layer_boundary.py tests/test_api_docs.py tests/test_api_reference.py tests/test_docs_examples.py tests/test_changelog_fragments.py-> 133 passedpython -m tools.changelog_fragments check-> OKpython -m tools.gen_api_docs --check-> OKNot run:
pytest -q-- still running locally at the tip; CI runs it on 3.9-3.13mkdocs build --strict-- nodocs/ormkdocs.ymlchange