Skip to content

feat: onboard a repository CI gate through a reviewed, human-confirmed pull request #543

Closed

Description

@HugoXK

Written-by: HugoXK

Problem

Fledge's auto-merge and review gate trust the repository's checks. But Fledge cannot help a repository get a trustworthy gate in the first place. A new repository either already has a good CI contract or auto-merge remains effectively unusable.

There is also no shared lane vocabulary: a PR-ready lane, an after-merge lane, a daily lane and a release lane are not distinguished, so Fledge cannot tell whether a green check is the right check at the right time.

Expected

  • A gate-onboarding job that runs on the repository and does read-only analysis first:
    • languages, package managers and the existing CI/workflow files;
    • how dependencies are installed, built, linted and tested;
    • which commands are fast enough for a PR, which need secrets/services/resources, and which are benchmarks or end-to-end;
    • whether a merge queue exists and which checks it requires;
    • existing ci-gates.json/release.yml if present.
  • Propose four lanes with commands, evidence and measured duration:
    • pr-ready: every PR commit; build, type check, lint, fast unit tests;
    • pr-merge: merge queue or after-merge; full tests, integration, secret-requiring checks;
    • daily: scheduled full/benchmark/audit runs, observe-only by default;
    • release: tag-triggered pre-publish verification.
  • The FlyBot agent may trial commands in its own workspace and report what it ran and how long it took. Fledge itself still does not clone or execute the repository.
  • A human reviews and may edit the proposed lanes. Nothing is written to the repository before confirmation.
  • After confirmation, Fledge creates exactly one gate pull request containing the gate job, .github/ci-gates.json, scripts/gate/gate.mjs when missing, and a tag-triggered release.yml when missing.
  • Existing workflow files are never replaced wholesale. Only the planned gate job, its required trigger events, ci-gates.json and release.yml are merged; unrelated jobs and triggers are preserved.
  • The PR records the default-branch SHA it was based on. If that branch moved after analysis, Fledge refuses and asks for a fresh run instead of writing a stale plan.
  • An empty lane runs as not_applicable, never as a pass. A lane that fails in observe mode is recorded as observe_fail, not hidden.
  • The repository page shows the installed lane contract and the last result per lane.

Red test

  • Planning performs no repository write; a confirmed plan creates one PR.
  • A branch that moved after analysis is refused.
  • An existing ci.yml keeps its unrelated jobs and triggers, and gains the gate job/event.
  • An empty lane reports not_applicable, not success.
  • An observe lane failure does not fail the gate job but is recorded.
  • A merge-queue repository gets merge_group; a non-merge-queue repository gets the after-merge path.
  • A check that runs on the merge preview instead of the PR head is reported as a gap.

Scope and tier

src/jobs/service.ts, src/repos/github.ts, src/lib/gate.ts (new), src/db.ts, src/views/pages.ts, tests, docs/architecture.md, docs/roadmap.md. T3, PASS at the tip.

Non-goals

  • No automatic merge of the gate pull request.
  • No overwrite of an existing workflow file.
  • No publishing to PyPI, npm or an image registry.
  • No arbitrary shell execution by Fledge; trial runs belong to the FlyBot agent and are reported as evidence.
  • No change to the existing review gate; this gives it a repository contract to trust.

Prior art

  • agent-ci-infra/bot docs/onboarding-guide.md and src/projects/gate-*: lane definitions, trial runs, human-confirmed plan, one gate PR, merge into existing files without replacement, base-SHA re-check, not_applicable and observe.
  • agent-ci-infra/bot docs/automation.md: pure planning modules, fail-closed facts, dry-run before execution.

Relations

Read at main 94972633569ea980483748df2ba1b1e84383d0ef.

Activity

  1. added
    enhancementNew feature or request
    area:jobsJob lifecycle, queue, callbacks
    needs-humanNeeds an owner or operator decision/action
    priority:highHigh priority; claim before normal backlog
    needs-reviewWaiting for the Fledge Platform Reviewer's issue review
    on Nov 13, 2026
  2. self-assigned this
    on Nov 13, 2026
  3. added
    priority:ownerOwner-planned task; prioritize ahead of the Product backlog
    on Nov 13, 2026
  4. HugoXK commented on Nov 13, 2026

    @HugoXK
    OwnerAuthor

    Claim

    Claimed by @HugoXK for implementation analysis. No branch yet.

    Current code reality

    • Fledge can verify an existing gate: verifyPullRequest(), path guard, check runs, commit status, reviews, branch protection, auto-merge reconciliation.
    • It cannot install or repair a gate. src/repos/github.ts has no file/blob/tree/commit/ref write methods and no ruleset read method. GithubClient.getBranchProtection() only reads branch protection.
    • The only write paths are createPullRequest() and comment(). There is no branch creation or multi-file commit.
    • Jobs has job kinds iterate|review|plan|scope|verify; there is no onboarding work type.
    • There is no YAML parser dependency in package.json; reading or editing .github/workflows/*.yml needs one.
    • Fledge never executes the repository. The FlyBot agent may run commands in its own workspace; Fledge itself must not.

    Target design

    Flow

    1. A maintainer starts a gate.onboarding job for a repository.
    2. The job analyzes the default branch read-only: languages, package managers, existing workflows, ci-gates.json, release.yml, branch protection/rulesets, merge queue, current check names.
    3. The FlyBot agent may trial commands in its own workspace and reports command, exit code and duration. Fledge records that as evidence.
    4. The agent returns a GatePlan in the callback:
      • pr-ready, pr-merge, daily, release lanes;
      • per lane: commands, source (existing script/Makefile/package script), observe flag, measured duration, notes;
      • existing gaps and risks;
      • the base SHA and the existing files it read.
    5. Fledge validates the plan structurally, stores it, renders a preview, and waits.
    6. A human edits/confirms via the management command (feat: add one management service contract for console, API and chat #540) or a form.
    7. Only after confirmation does Fledge create one branch and one PR with the gate files. The base SHA is re-checked; if it moved, refuse and ask for a fresh run.
    8. The gate PR is merged by a human; Fledge never auto-merges it.
    9. After merge, Fledge verifies the gate on the next PR and can use it for auto-merge.

    Schema

    CREATE TABLE gate_sessions (
      id TEXT PRIMARY KEY,
      repo_id TEXT NOT NULL REFERENCES repos(id) ON DELETE CASCADE,
      status TEXT NOT NULL,              -- analyzing|proposed|confirmed|creating|created|refused|failed
      base_sha TEXT NOT NULL,
      plan_json TEXT,
      plan_hash TEXT,
      confirmed_by TEXT,
      confirmed_at TEXT,
      pr_url TEXT,
      branch TEXT,
      error_code TEXT,
      created_at TEXT NOT NULL,
      updated_at TEXT NOT NULL
    );

    plan_hash makes the confirmed bytes immutable. A new plan or a moved base SHA invalidates the confirmation.

    Plan schema

    {
      "baseSha": "40-hex",
      "lanes": {
        "pr-ready": {"commands": ["npm ci", "npm run check"], "observe": false, "source": "package.json", "durationMs": 180000},
        "pr-merge": {"commands": ["npm run test:integration"], "observe": false, "source": "scripts/test.sh", "durationMs": 900000},
        "daily": {"commands": ["npm run test:e2e"], "observe": true, "source": "existing nightly workflow", "durationMs": 2400000},
        "release": {"commands": ["npm run build"], "observe": false, "source": "release.yml", "durationMs": 300000}
      },
      "mergeQueue": true,
      "existingFiles": [".github/workflows/ci.yml"],
      "gaps": ["no lint job", "tests run on merge preview instead of PR head"],
      "notes": ["commands were trial-run at <sha>"]
    }

    Validate: lane names closed set; command strings non-empty and length-bounded; no shell metacharacter injection into a generated YAML run: block beyond a plain string; observe boolean; baseSha 40-hex; no unknown fields.

    Generated files

    • .github/ci-gates.json: lanes, commands, observe, events mapping.
    • .github/workflows/ci-gates.yml: gate job. Triggers depend on facts:
      • pull_request for pr-ready;
      • merge_group when a merge queue exists;
      • push to the default branch for the after-merge path when no merge queue;
      • schedule for daily only when lanes have commands;
      • workflow_dispatch for manual.
    • scripts/gate/gate.mjs: a small, original gate runner with not_applicable, observe_fail, fail, JSON report and no GitHub token for lane commands.
    • .github/workflows/release.yml: tag trigger that runs the release lane.

    Use the yaml package (the same parser family the upstream bot uses) via parseDocument and stringify. Do not hand-roll YAML.

    Existing workflow merge

    When the plan names an existing workflow:

    • parse it with parseDocument;
    • preserve every unrelated job and trigger;
    • if a gate job already exists, replace only that job; otherwise add gate;
    • add only the missing trigger events;
    • if other jobs exist and extra triggers would make them run on gate events, add an if: guard to the gate job and explain it in the PR body;
    • never delete an unrelated job or trigger;
    • if the file cannot be parsed unambiguously, refuse and create a new ci-gates.yml instead.

    The PR body lists the diff, the facts used, commands not run, and the known gaps.

    GitHub writes

    Add to GithubClient:

    • getFile(owner, name, path, ref) -> { sha, content }
    • createBlob, createTree, createCommit, createRef, updateRef
    • listRulesets, getBranchProtection (exists), getRequiredChecks
    • createPullRequest (exists)

    The commit is built from the base tree plus only the planned files, then one branch fledge/gate-<session> and one PR. No force push. A 422 from an existing branch is a refusal, not a retry.

    Human preview

    Render:

    • full content for new files;
    • a unified diff for modified files (add a small line-diff helper or a pinned diff dependency; do not render raw before/after without a diff);
    • lane table with source, observed duration, observe flag and warnings.

    The confirmation form includes expectedRevision/plan_hash and baseSha. A change to either returns a conflict, not a silent overwrite.

    Implementation slices

    1. GitHub file/commit client + tests. Fake server, blob/tree/commit/ref, existing branch refusal.
    2. Plan schema + validation + storage. gate_sessions, plan hash, unit tests.
    3. Onboarding job + prompt + callback. New job kind, trial evidence, no writes.
    4. YAML generation/merge. ci-gates.json, ci-gates.yml, gate.mjs, release.yml, preservation tests.
    5. Preview + confirm + create PR. Management command (feat: add one management service contract for console, API and chat #540), base-SHA re-check, one PR.
    6. UI + post-merge verification. Gate status, not_applicable display, auto-merge readiness.

    Failure, concurrency and security

    • No repository write before a confirmed plan and a matching base SHA.
    • The plan is a closed schema; reject unknown fields and unknown lane names.
    • The generated workflow must not interpolate untrusted PR titles/bodies/branch names into a shell run: block. Use environment variables or fixed commands.
    • Lane commands run without a GitHub token. The gate job's own token is scoped to reading the run.
    • The gate PR is one commit, one branch, one PR; a new confirmation with a different plan hash needs a new branch or an explicit update, never a force push.
    • A moved base branch is a refusal (gate_base_changed), not a merge onto a stale base.
    • Do not accept a plan that claims a command passed without trial evidence; mark it unverified.
    • Keep the current Figure/UI accessibility contract: the preview is keyboard reachable and the confirm action names what it will write.

    Observability

    • fledge_gate_sessions_total{status}.
    • fledge_gate_plan_lanes{lane}.
    • Audit gate.plan.proposed/confirmed/created/refused.
    • Job events gate.analyzed, gate.confirmed, gate.pr_created.

    Test plan

    • No write before confirmation; one write after.
    • Moved base SHA refuses.
    • Existing ci.yml keeps unrelated jobs/triggers and gains the gate job.
    • Merge-queue repo gets merge_group; non-queue repo gets the after-merge path.
    • Empty lane -> not_applicable, not success.
    • Observe failure -> observe_fail, gate job green, recorded.
    • A plan with an unknown field or a missing lane is refused.
    • A branch that already exists is refused, not overwritten.
    • Gate PR diff contains exactly the planned files.

    Risk and dependencies

    Open decisions

    1. Do we support editing an existing workflow in v1, or only add a new ci-gates.yml? Recommend support only the planned insertion with a refusal when parsing is ambiguous.
    2. Do we trial-run commands at all in v1? Recommend yes through the FlyBot agent, with evidence; never through Fledge.
    3. Is the gate PR opened against the default branch directly? Recommend yes, as a normal PR, never auto-merged.
  5. HugoXK-bot commented on Nov 13, 2026

    @HugoXK-bot
    Collaborator

    Issue-review: PASS at issue #543 body a3f2b870ac57
    Written-by: HugoXK-bot
    Posted-by: HugoXK-bot

    Ran: live body read this turn — raw=a3f2b870ac57, trimmed=aa77ca1fade7, 4,302 chars, updated_at 2026-11-13T07:31:42Z, labels enhancement, area:jobs, needs-human, needs-review, area:repos, priority:owner, priority:high. At live main 949726335: .github/ci-gates.json and scripts/gate/gate.mjs do not exist (so "when missing" is accurate), and no src/lib/gate.ts exists. No duplicate card among the 23 open issues.
    Not run: no suite — a card.

    1. issue body:20 — "The FlyBot agent may trial commands in its own workspace and report what it ran and how long it took." Say that the trial output is evidence attached to the plan, never a gate input or a Fledge-side execution, so the AGENTS rule "Fledge never executes user code" stays visible at the point a dev would be tempted to add a runner.
    2. issue body:38 — the scope names Fledge paths only, but the deliverable and the red tests are about files written into the target repository. Name the Fledge-side renderer (src/lib/gate.ts) and note that the write goes through the App token as one PR, so the "one PR, unrelated jobs preserved" test has a module to point at.

    Conclusion: PASS. The plan/confirm split, the base-SHA re-check, not_applicable and observe_fail, and "never replace an existing workflow wholesale" are the properties that make this safe, and needs-human is right because the confirmation step is the product.

  6. added
    review:approvedIssue reviewed and approved by the Fledge Platform Reviewer
    and removed
    needs-reviewWaiting for the Fledge Platform Reviewer's issue review
    on Nov 13, 2026
  7. added a commit that references this issue on Nov 14, 2026
  8. HugoXK commented on Nov 14, 2026

    @HugoXK
    OwnerAuthor

    Slice 1 is open: #612.

    Implemented: closed GatePlan schema + validation, gate_sessions storage, maintainer-only create/get/confirm API, preview UI, tests.

    Not in this PR: GitHub file/commit writes, gate PR creation, workflow YAML merge, trial runs.

  9. 21 remaining items

  10. added
    enhancementNew feature or request
    area:jobsJob lifecycle, queue, callbacks
    review:approvedIssue reviewed and approved by the Fledge Platform Reviewer
    priority:ownerOwner-planned task; prioritize ahead of the Product backlog
    priority:highHigh priority; claim before normal backlog
    and removed
    priority:ownerOwner-planned task; prioritize ahead of the Product backlog
    priority:highHigh priority; claim before normal backlog
    on Nov 16, 2026
  11. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Dev-2

    Claimed. Branch dev2/543-ci-gate-onboard, cut on the live main I read in the turn of the cut.

    Slice 1 is the planner and the stored plan, no repository write: src/lib/gate.ts (new) turns read-only repository facts into the four lanes (pr-ready, pr-merge, daily, release) with commands, evidence and measured duration, and decides merge_group versus the after-merge path; a stored plan records the default-branch SHA it was based on and a confirmation after that branch moved is refused with a fresh-run request rather than a stale plan. The gate pull request creation, the in-workspace trial runs and the repository-page result per lane stay the next slices and are not in this diff.

    Red first on the branch commit before the change: planning writes nothing, a moved branch is refused, an existing ci.yml keeps its unrelated jobs and triggers, an empty lane is not_applicable, an observe lane failure is recorded rather than hidden, a merge-queue repository gets merge_group and a non-merge-queue repository gets the after-merge path.

    Label note: my first label write used the whole-set endpoint with only agent-working and cleared this issue's labels for a moment; they are restored in the same minute (enhancement, area:jobs, area:repos, review:approved, priority:owner, priority:high, plus agent-working, with agent-ready dropped as the claim requires). The body and every verdict are untouched.

  12. added
    agent-readyReady for an agent to implement
    and removed on Nov 16, 2026
  13. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Dev-2

    Claim released, no branch was cut.

    This card is already recorded obsolete at comment 6894707487 (obsolete -> #830), and the code agrees: src/gates/plan.ts, src/gates/lanes.ts, src/gates/compose.ts and src/gates/apply.ts are on main from PR #830 (7b272bbd2fc6a93d03168de3793fdab5a773bfeb), with the read-only report behind GET /repos/:id/ci-gate and src/lib/ciGate.ts. The four lanes, merge_group versus the after-merge path, not_applicable, observe_fail, the plan/confirm flow and the one-gate-PR write are implemented there, so anything I built here would be a second copy.

    Claim state restored: agent-working off, agent-ready back on, every other label untouched. Nothing for me to do on this card; the close belongs to the owner.

  14. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Lead

    Closing as delivered, with the shas this card's scope landed under. Nothing here was cut as a branch; every slice is already on main.

    • a172931afa07d85b255b279fa2f026d21437538c (2026-11-14T04:26:21Z, "feat: add gate plan schema, sessions and human confirmation (feat: add gate plan schema, sessions and human confirmation #612)") — src/gates/plan.ts: the plan schema, a stored plan bound to the default-branch SHA it was based on, and the human confirmation that refuses a plan whose branch moved.
    • a1d0978ff2d4f86ca2555db07b7848011b60da07 (2026-11-15T05:25:56Z, "feat: report whether a repository own CI gates a merge (feat: onboard a repository CI gate through a reviewed, human-confirmed pull request #543)") — src/lib/ciGate.ts, src/views/ciGate.ts, and the read-only GET /repos/:id/ci-gate route: the analysis/report half, answering whether the repository's own CI gates a merge.
    • 7b272bbd2fc6a93d03168de3793fdab5a773bfeb (2026-11-15T10:54:17Z, squash of PR feat(gates): open one branch and one pull request per confirmed gate #830 "feat(gates): open one branch and one pull request per confirmed gate") — src/gates/apply.ts, src/gates/compose.ts, src/gates/lanes.ts, src/views/gate.ts, plus src/app.ts (+114) with POST /repos/:id/gate/sessions, POST /repos/:id/gate/sessions/:sessionId/confirm and the applied gate PR: the one branch and one pull request per confirmed plan.

    Read on live main 24294731a38c541da610b4cc07b6137b7a86a0fc just before this comment: src/gates/plan.ts, lanes.ts, compose.ts, apply.ts, src/lib/ciGate.ts, src/views/ciGate.ts all present; applyGatePlan() called at src/app.ts:2849; the four lanes, merge_group versus the after-merge path, not_applicable, the plan/confirm flow and the one-gate-PR write are all implemented there.

    So a branch for this card would duplicate #830. Disposition: obsolete -> #830, closed as completed. Prior claim without a branch was already released (comment 6894707487); the second claim (Dev-2, 03:39Z) was released at 03:42Z with no branch cut. Nothing is left holding this card.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

agent-readyReady for an agent to implementarea:jobsJob lifecycle, queue, callbacksarea:reposenhancementNew feature or requestpriority:highHigh priority; claim before normal backlogpriority:ownerOwner-planned task; prioritize ahead of the Product backlogreview:approvedIssue reviewed and approved by the Fledge Platform Reviewer

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions