diff --git a/README.md b/README.md index be387bf6..27f65101 100644 --- a/README.md +++ b/README.md @@ -94,13 +94,13 @@ The fully unattended `mergesutra run` command is **not shipped**. Neither is an | Claim | Inspectable evidence | Honest boundary | | --- | --- | --- | -| Cross-platform engineering | [Six CI jobs passed at `c6fd871`](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42492506208) on Linux, Windows and macOS with Node 22 and Node 24 | Not a live BharatCode result | -| Working customer CLI stages | [Eight hosted scripted customer journeys passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42489888186) on PR #70's pre-merge revision | Scripted gateway; not the final merged SHA | +| Cross-platform engineering | [Six CI jobs passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) on merged `a381ff0`, across Linux, Windows and macOS with Node 22 and Node 24 | Not a live BharatCode result | +| Working customer CLI stages | [Eight hosted scripted journeys passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) on merged `a381ff0` | Scripted gateway; not a live model | | Real BharatCode used successfully | [Historical Qwen fixture at `b1c18bf`](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041), 11 October: 9/12 model requests, independent acceptance passed | Older commit, one bounded fixture; no published PR | | Isolated repository execution | [Linux Docker sandbox jobs passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41614337044) | Some host/command combinations remain unqualified | | Representative AI reliability | [Ten-case benchmark ledger](https://github.com/Pavithran-R-A/mergesutra/blob/main/docs/REAL_TASK_BENCHMARK.md) | **0/10 live cases recorded** | -**Still unproven:** Full live BharatCode qualification on the present submission SHA, the ten-case reliability benchmark, real remote draft-PR publication, and macOS Docker-host behavior. The writer retains a documented final-check-to-rename race. These limits are disclosed—not hidden behind a 100% claim. +**Still unproven:** Full live BharatCode qualification on `a381ff0`, the ten-case reliability benchmark, real remote draft-PR publication, and macOS Docker-host behavior. The writer retains a documented final-check-to-rename race. These limits are disclosed—not hidden behind a 100% claim. ## Architecture: where trust is enforced diff --git a/docs/JUDGE_GUIDE.md b/docs/JUDGE_GUIDE.md index 6b8f5951..ae5e37c7 100644 --- a/docs/JUDGE_GUIDE.md +++ b/docs/JUDGE_GUIDE.md @@ -23,10 +23,11 @@ A single run can be inspected through the shipped commands: | Question | Evidence | Precise reading | | --- | --- | --- | -| Does the source build and install across supported Node versions? | [CI run #42481055084](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42481055084) | Six green jobs on Ubuntu, Windows and macOS × Node 22 and 24 at `e61fd5d` (8 Nov 2026) | -| Does the production CLI complete customer journeys? | [Customer workflow run #42478877591](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42478877591) | Eight hosted journeys passed on PR #69 head `55f1351`; **scripted-model gateway**, not a live model or the exact post-merge SHA | +| Does the source build and install across supported Node versions? | [Merged-main CI run #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | Six green jobs on Ubuntu, Windows and macOS × Node 22 and 24 at `a381ff0` (10 Nov 2026) | +| Does the production CLI complete customer journeys? | [Merged-main customer run #42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) | Eight hosted journeys passed at `a381ff0`; **scripted-model gateway**, not a live model | | Are commands isolated? | [Docker isolation run #41614337044](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41614337044) | Three green Linux Docker isolation jobs, within the documented image/host/command limits | | Has BharatCode actually completed a bounded issue-to-patch task? | [Live Qwen run #38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041) | Successful **older** 11 Oct 2026 fixture run at `b1c18bf`: 9/12 model requests; independent acceptance passed; **no PR published** | +| Did the latest full live qualification pass? | [Failed run #42497165107](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42497165107) | No. Qwen planning timed out at older `801aab0`; no implementation or review followed | | Does the present adapter answer at least one real request? | [Live smoke #41393924176](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41393924176) | A bounded single request passed on 2 Nov 2026; this is **not** a full live chain | | Has the ten-case independent real-task benchmark passed? | [Benchmark protocol and ledger](REAL_TASK_BENCHMARK.md) | **No. 0/10 live-model cases recorded.** Offline fixtures and negative controls exist | diff --git a/docs/SUBMISSION.md b/docs/SUBMISSION.md index 519b5f1a..747e4b51 100644 --- a/docs/SUBMISSION.md +++ b/docs/SUBMISSION.md @@ -12,9 +12,9 @@ submission has been made. | Public GitHub repository | PASS | `gh repo view Pavithran-R-A/mergesutra` reports `"visibility":"PUBLIC"` | | `BharatCode.txt` at the repository root | PASS | `gh api repos/Pavithran-R-A/mergesutra/contents/BharatCode.txt` returns the file at `main` | | Built with BharatCode | PASS, with stated limits | Live run [#38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041), 11 October 2026, `qwen-3.8-27b`, commit `b1c18bfaf299dfae6316b70e17647a6bde48caee` | -| Working project | PASS | `npm run build`; completed scripted-model journey `run-20261021T115401Z-ea10b8` (2/2 gates, 5/5 oracle checks, 0 review findings); hosted CI on `main` | +| Working project | PASS | Merged-main CI [#42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458), scripted customer workflow [#42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193), plus the 21 October scripted demo | | Submission form fields | READY | Name, URL and summary below | -| Submission made | **NO** | No account-linked entry exists until the organiser's site confirms one | +| Submission made | **OWNER-REPORTED; NOT RECHECKED** | Issue #72 records the owner's screenshot; this environment cannot inspect account-linked status | | Deadline | 10 November 2026, 11:59 PM IST | Must be submitted from the organiser's site by the account holder; re-verified on on 23 October 2026 | Eligibility: students and other builders. Judging: GitHub stars, website votes, diff --git a/docs/releases/2026-11-10-qualification.md b/docs/releases/2026-11-10-qualification.md new file mode 100644 index 00000000..7ef95bc8 --- /dev/null +++ b/docs/releases/2026-11-10-qualification.md @@ -0,0 +1,138 @@ +# MergeSutra release qualification — 10 November 2026 + +## Verdict + +**BLOCKED for full release qualification.** The code candidate passes +deterministic, package, artifact, path, cross-platform, and scripted customer +gates. The current-SHA live model run remains unexecuted. The independent +ten-task benchmark remains `NOT_EXECUTED` (0/10). No model requests were spent +for this qualification. + +This is not a production-readiness claim. Submission assets are technically +ready. The account holder must verify contest-entry status before the deadline. + +## Qualified revision + +- Final `main`: `a381ff064873f631da7bc86783d2a197fb03af8b`. +- PR #77 merged as `fda2615c03c856debcf2bea9bc63da9f47b367ab`. +- PR #78 merged as `a381ff064873f631da7bc86783d2a197fb03af8b`. +- PR #78 head: `a48f2d53b61dc35a69ed58d951315630b80d5106`. +- PR #78 was retargeted to `main`. +- Its update preserved remote history. +- It was merged normally after review. + +## Hosted evidence + +| Gate | Result | Evidence | +| --- | --- | --- | +| Ubuntu, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| Ubuntu, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| Windows, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| Windows, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| macOS, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| macOS, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | +| Eight scripted customer journeys | PASS | [Main #42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) | +| PR-head matrix and customer gate | PASS | [CI #42675078631](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42675078631), [customer #42675078865](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42675078865) | + +The PR-head jobs tested `a48f2d5`. +The merged-main jobs tested `a381ff0`. +The customer workflow uses a scripted model. + +## Local release gates + +Windows 10.0.26200; Node `v24.22.2`; npm `11.19.1`. +The working directory was the PR #78 worktree. +TMP/TEMP resolved to the user temp directory. +Vitest used repository defaults without overrides. + +| Command | Result | +| --- | --- | +| `npm ci` | PASS | +| `npm run check` | PASS: format, lint, types, build; 2,997 passed, 12 skipped | +| `npm run qualify:path-shapes` | PASS: Windows path cases | +| `npm run verify:package` | PASS: 115 tests | +| `npm run test:artifact` | PASS: 15 tests | +| `npm audit --omit=dev --audit-level=high` | PASS: zero production vulnerabilities | +| `docker version --format '{{.Server.Version}}'` | BLOCKED locally: Docker daemon absent | + +Hosted customer qualification passed in Docker. +The Windows-local result supplements hosted matrix evidence. +The 12 skipped tests remain disclosed. + +## Confirmed corrections + +PR #78 initially failed formatting. +`scripts/e2e-validation.mjs` had a multiline import Prettier rejected. +The import was formatted with the pinned formatter. + +TypeScript then found missing declarations for two `.mjs` modules. +Added local declarations and explicit test callback types. +Updated source-shape assertions for both fixture profiles. +The assertions now check the actual profile routing. + +The full local run had four failures. +Two were stale source-shape assertions, now corrected. +The timing oracle passed its focused rerun. +It also passed during the full rerun. +Its 45-second guard was not raised. +The Git fixture timeout passed focused and full reruns. +No product timeout was increased or suppressed. + +Files changed by PR #78: + +- `.github/workflows/live-validation.yml` +- `scripts/e2e-validation.mjs` +- `scripts/lib/compact-qualification.mjs` +- `scripts/lib/compact-qualification.d.mts` +- `scripts/lib/live-qualification.mjs` +- `scripts/lib/live-qualification.d.mts` +- `tests/security/compact-live-qualification.test.ts` +- `tests/security/live-validation-workflow-policy.test.ts` + +## Security and model qualification + +The live workflow remains manual-dispatch only. +The live job requires `refs/heads/main` and `run_live=true`. +The selected model must appear in the authenticated catalog. +Repository commands remain inside the pinned Docker sandbox. +Repository commands receive no model credential. +The completion ceiling remains 12. +Automatic retries remain zero. +The workflow rejects rerun attempts before credentials are used. +Artifacts contain sanitized driver exports only. +The workflow has no PR, push, or merge step. + +The compact profile uses public fixture issue #1. +Its independent oracle remains outside model write scope. +Regression tests reject incomplete assertions and forbidden paths. +They also reject dependencies and a failing oracle. +Compact evidence cannot satisfy full-profile path requirements. + +No live workflow was dispatched here. +No BharatCode request or paid infrastructure was used. + +Historical live evidence remains separate: + +- [Successful Qwen run #38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041) used old SHA `b1c18bf`; it consumed 9/12 requests and passed its independent fixture check. +- [Full run #42497165107](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42497165107) failed planning on `801aab0`; no implementation or review followed. +- The newer compact profile has no live result. +- The ten-task benchmark remains `NOT_EXECUTED`, 0/10. + +## Submission and remaining limits + +- Repository visibility: PUBLIC. +- `BharatCode.txt`: present on `main`. +- The 88-second demo link returned HTTP 200. +- The demo remains explicitly scripted, not live. +- No separate project-overview video was found. +- The official rules document no video requirement. +- The owner reported an existing contest entry. +- This environment cannot inspect account-linked status. +- The npm package remains unpublished. +- macOS Docker-host behavior remains unqualified. +- The writer retains a final-check-to-rename race. +- Real remote draft-PR publication remains unqualified. +- Full profile on this exact SHA remains unqualified. + +Verify the existing entry at the +[official Build League page](https://bharatcode.ai/hackathon).