Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,13 +94,13 @@ The fully unattended `mergesutra run` command is **not shipped**. Neither is an

| Claim | Inspectable evidence | Honest boundary |
| --- | --- | --- |
| Cross-platform engineering | [Six CI jobs passed at `c6fd871`](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42492506208) on Linux, Windows and macOS with Node 22 and Node 24 | Not a live BharatCode result |
| Working customer CLI stages | [Eight hosted scripted customer journeys passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42489888186) on PR #70's pre-merge revision | Scripted gateway; not the final merged SHA |
| Cross-platform engineering | [Six CI jobs passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) on merged `a381ff0`, across Linux, Windows and macOS with Node 22 and Node 24 | Not a live BharatCode result |
| Working customer CLI stages | [Eight hosted scripted journeys passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) on merged `a381ff0` | Scripted gateway; not a live model |
| Real BharatCode used successfully | [Historical Qwen fixture at `b1c18bf`](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041), 11 October: 9/12 model requests, independent acceptance passed | Older commit, one bounded fixture; no published PR |
| Isolated repository execution | [Linux Docker sandbox jobs passed](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41614337044) | Some host/command combinations remain unqualified |
| Representative AI reliability | [Ten-case benchmark ledger](https://github.com/Pavithran-R-A/mergesutra/blob/main/docs/REAL_TASK_BENCHMARK.md) | **0/10 live cases recorded** |

**Still unproven:** Full live BharatCode qualification on the present submission SHA, the ten-case reliability benchmark, real remote draft-PR publication, and macOS Docker-host behavior. The writer retains a documented final-check-to-rename race. These limits are disclosed—not hidden behind a 100% claim.
**Still unproven:** Full live BharatCode qualification on `a381ff0`, the ten-case reliability benchmark, real remote draft-PR publication, and macOS Docker-host behavior. The writer retains a documented final-check-to-rename race. These limits are disclosed—not hidden behind a 100% claim.

## Architecture: where trust is enforced

Expand Down
5 changes: 3 additions & 2 deletions docs/JUDGE_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ A single run can be inspected through the shipped commands:

| Question | Evidence | Precise reading |
| --- | --- | --- |
| Does the source build and install across supported Node versions? | [CI run #42481055084](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42481055084) | Six green jobs on Ubuntu, Windows and macOS × Node 22 and 24 at `e61fd5d` (8 Nov 2026) |
| Does the production CLI complete customer journeys? | [Customer workflow run #42478877591](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42478877591) | Eight hosted journeys passed on PR #69 head `55f1351`; **scripted-model gateway**, not a live model or the exact post-merge SHA |
| Does the source build and install across supported Node versions? | [Merged-main CI run #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) | Six green jobs on Ubuntu, Windows and macOS × Node 22 and 24 at `a381ff0` (10 Nov 2026) |
| Does the production CLI complete customer journeys? | [Merged-main customer run #42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) | Eight hosted journeys passed at `a381ff0`; **scripted-model gateway**, not a live model |
| Are commands isolated? | [Docker isolation run #41614337044](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41614337044) | Three green Linux Docker isolation jobs, within the documented image/host/command limits |
| Has BharatCode actually completed a bounded issue-to-patch task? | [Live Qwen run #38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041) | Successful **older** 11 Oct 2026 fixture run at `b1c18bf`: 9/12 model requests; independent acceptance passed; **no PR published** |
| Did the latest full live qualification pass? | [Failed run #42497165107](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42497165107) | No. Qwen planning timed out at older `801aab0`; no implementation or review followed |
| Does the present adapter answer at least one real request? | [Live smoke #41393924176](https://github.com/Pavithran-R-A/mergesutra/actions/runs/41393924176) | A bounded single request passed on 2 Nov 2026; this is **not** a full live chain |
| Has the ten-case independent real-task benchmark passed? | [Benchmark protocol and ledger](REAL_TASK_BENCHMARK.md) | **No. 0/10 live-model cases recorded.** Offline fixtures and negative controls exist |

Expand Down
4 changes: 2 additions & 2 deletions docs/SUBMISSION.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ submission has been made.
| Public GitHub repository | PASS | `gh repo view Pavithran-R-A/mergesutra` reports `"visibility":"PUBLIC"` |
| `BharatCode.txt` at the repository root | PASS | `gh api repos/Pavithran-R-A/mergesutra/contents/BharatCode.txt` returns the file at `main` |
| Built with BharatCode | PASS, with stated limits | Live run [#38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041), 11 October 2026, `qwen-3.8-27b`, commit `b1c18bfaf299dfae6316b70e17647a6bde48caee` |
| Working project | PASS | `npm run build`; completed scripted-model journey `run-20261021T115401Z-ea10b8` (2/2 gates, 5/5 oracle checks, 0 review findings); hosted CI on `main` |
| Working project | PASS | Merged-main CI [#42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458), scripted customer workflow [#42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193), plus the 21 October scripted demo |
| Submission form fields | READY | Name, URL and summary below |
| Submission made | **NO** | No account-linked entry exists until the organiser's site confirms one |
| Submission made | **OWNER-REPORTED; NOT RECHECKED** | Issue #72 records the owner's screenshot; this environment cannot inspect account-linked status |
| Deadline | 10 November 2026, 11:59 PM IST | Must be submitted from the organiser's site by the account holder; re-verified on <https://bharatcode.ai/hackathon> on 23 October 2026 |

Eligibility: students and other builders. Judging: GitHub stars, website votes,
Expand Down
138 changes: 138 additions & 0 deletions docs/releases/2026-11-10-qualification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# MergeSutra release qualification — 10 November 2026

## Verdict

**BLOCKED for full release qualification.** The code candidate passes
deterministic, package, artifact, path, cross-platform, and scripted customer
gates. The current-SHA live model run remains unexecuted. The independent
ten-task benchmark remains `NOT_EXECUTED` (0/10). No model requests were spent
for this qualification.

This is not a production-readiness claim. Submission assets are technically
ready. The account holder must verify contest-entry status before the deadline.

## Qualified revision

- Final `main`: `a381ff064873f631da7bc86783d2a197fb03af8b`.
- PR #77 merged as `fda2615c03c856debcf2bea9bc63da9f47b367ab`.
- PR #78 merged as `a381ff064873f631da7bc86783d2a197fb03af8b`.
- PR #78 head: `a48f2d53b61dc35a69ed58d951315630b80d5106`.
- PR #78 was retargeted to `main`.
- Its update preserved remote history.
- It was merged normally after review.

## Hosted evidence

| Gate | Result | Evidence |
| --- | --- | --- |
| Ubuntu, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| Ubuntu, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| Windows, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| Windows, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| macOS, Node 22 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| macOS, Node 24 | PASS | [Main CI #42677315458](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677315458) |
| Eight scripted customer journeys | PASS | [Main #42677463193](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42677463193) |
| PR-head matrix and customer gate | PASS | [CI #42675078631](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42675078631), [customer #42675078865](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42675078865) |

The PR-head jobs tested `a48f2d5`.
The merged-main jobs tested `a381ff0`.
The customer workflow uses a scripted model.

## Local release gates

Windows 10.0.26200; Node `v24.22.2`; npm `11.19.1`.
The working directory was the PR #78 worktree.
TMP/TEMP resolved to the user temp directory.
Vitest used repository defaults without overrides.

| Command | Result |
| --- | --- |
| `npm ci` | PASS |
| `npm run check` | PASS: format, lint, types, build; 2,997 passed, 12 skipped |
| `npm run qualify:path-shapes` | PASS: Windows path cases |
| `npm run verify:package` | PASS: 115 tests |
| `npm run test:artifact` | PASS: 15 tests |
| `npm audit --omit=dev --audit-level=high` | PASS: zero production vulnerabilities |
| `docker version --format '{{.Server.Version}}'` | BLOCKED locally: Docker daemon absent |

Hosted customer qualification passed in Docker.
The Windows-local result supplements hosted matrix evidence.
The 12 skipped tests remain disclosed.

## Confirmed corrections

PR #78 initially failed formatting.
`scripts/e2e-validation.mjs` had a multiline import Prettier rejected.
The import was formatted with the pinned formatter.

TypeScript then found missing declarations for two `.mjs` modules.
Added local declarations and explicit test callback types.
Updated source-shape assertions for both fixture profiles.
The assertions now check the actual profile routing.

The full local run had four failures.
Two were stale source-shape assertions, now corrected.
The timing oracle passed its focused rerun.
It also passed during the full rerun.
Its 45-second guard was not raised.
The Git fixture timeout passed focused and full reruns.
No product timeout was increased or suppressed.

Files changed by PR #78:

- `.github/workflows/live-validation.yml`
- `scripts/e2e-validation.mjs`
- `scripts/lib/compact-qualification.mjs`
- `scripts/lib/compact-qualification.d.mts`
- `scripts/lib/live-qualification.mjs`
- `scripts/lib/live-qualification.d.mts`
- `tests/security/compact-live-qualification.test.ts`
- `tests/security/live-validation-workflow-policy.test.ts`

## Security and model qualification

The live workflow remains manual-dispatch only.
The live job requires `refs/heads/main` and `run_live=true`.
The selected model must appear in the authenticated catalog.
Repository commands remain inside the pinned Docker sandbox.
Repository commands receive no model credential.
The completion ceiling remains 12.
Automatic retries remain zero.
The workflow rejects rerun attempts before credentials are used.
Artifacts contain sanitized driver exports only.
The workflow has no PR, push, or merge step.

The compact profile uses public fixture issue #1.
Its independent oracle remains outside model write scope.
Regression tests reject incomplete assertions and forbidden paths.
They also reject dependencies and a failing oracle.
Compact evidence cannot satisfy full-profile path requirements.

No live workflow was dispatched here.
No BharatCode request or paid infrastructure was used.

Historical live evidence remains separate:

- [Successful Qwen run #38144174041](https://github.com/Pavithran-R-A/mergesutra/actions/runs/38144174041) used old SHA `b1c18bf`; it consumed 9/12 requests and passed its independent fixture check.
- [Full run #42497165107](https://github.com/Pavithran-R-A/mergesutra/actions/runs/42497165107) failed planning on `801aab0`; no implementation or review followed.
- The newer compact profile has no live result.
- The ten-task benchmark remains `NOT_EXECUTED`, 0/10.

## Submission and remaining limits

- Repository visibility: PUBLIC.
- `BharatCode.txt`: present on `main`.
- The 88-second demo link returned HTTP 200.
- The demo remains explicitly scripted, not live.
- No separate project-overview video was found.
- The official rules document no video requirement.
- The owner reported an existing contest entry.
- This environment cannot inspect account-linked status.
- The npm package remains unpublished.
- macOS Docker-host behavior remains unqualified.
- The writer retains a final-check-to-rename race.
- Real remote draft-PR publication remains unqualified.
- Full profile on this exact SHA remains unqualified.

Verify the existing entry at the
[official Build League page](https://bharatcode.ai/hackathon).
Loading