Repository navigation
feat(ci): add Windows unit-test and scheduled-workflow liveness domains to the nightly T3 run - #2835
Conversation
![benchbox-oracle[bot]](https://avatars.githubusercontent.com/u/319264189?s=60&v=4){kind=small}
There was a problem hiding this comment.
oracle-review-shadow: failure for daab37524ca857a9ee4dbe00a0bf8b0ab055e39c
This result supersedes those for earlier heads; their open review threads still apply.
Tier: low-medium (data-only or small single-area change).
Decision: SHIP WITH FIXES.
Reviewer: luna (codex gpt-6-luna, effort high).
The new liveness domain can produce an incorrect verdict when manually dispatched from a non-default branch because its checkout may not match the workflow tree that generated scheduled runs.
There was a problem hiding this comment.
oracle-verdict: success for 3d902ba85d85ae855780dd8c725635934fdc04c8
This result supersedes those for earlier heads; their open review threads still apply.
Tier: low-medium (data-only or small single-area change).
Decision: SHIP.
Reviewer: luna (codex gpt-6-luna, effort high).
No merge-blocking defects found in the reviewed changes.
Verified fixed: D1. Resolve their review threads.
|
Stand-in oracle review: APPROVE 3d902ba |
3d902ba to
1c6dfca
Compare
There was a problem hiding this comment.
oracle-verdict: pending for 1c6dfcaa72553d59bed1de324efab79ccd246f95
This result supersedes those for earlier heads; their open review threads still apply.
Tier: low-medium (data-only or small single-area change).
Reviewers not available
- luna: absent (error)
- muse: absent (invalid)
- sol: absent (error)
- agy: disabled (CI authentication and quota output are not yet calibrated, and the workflow has no agy job)
- sonnet: skipped: same family as the author (claude)
There was a problem hiding this comment.
oracle-verdict: success for 1c6dfcaa72553d59bed1de324efab79ccd246f95
This result supersedes those for earlier heads; their open review threads still apply.
Tier: low-medium (data-only or small single-area change).
Decision: SHIP.
Reviewer: muse (muse muse-spark-1.3-contributor, effort medium).
Added Windows fast-tier nightly domain with pinned actions, read-only permissions, artifact upload, report integration, docs, and harness encoding fixes. No benchmark-correctness, verdict, gate-weakening, or security defect found.
Reviewers not available
- luna: absent (error)
…ns to the nightly T3 run The new nightly's Windows jobs only build the wheel and run the bundled data generators; the unit suite on Windows runs only in the legacy nightly, which also holds the only scheduled-workflow liveness check. Retiring the legacy workflow would have ended both without anyone deciding to, so the nightly T3 run gets them as domains of its own. - t3:windows runs the fast unit tier on windows-latest with Python 3.12 (no coverage gate, which the Linux job already enforces). Its 50 minute timeout is the measured 38 minute run plus a margin above 25%, and well under the platform default so a hung job fails in under an hour. - t3:liveness runs scripts/scheduled_workflow_liveness.py with read-only Actions access and keeps its log as an artifact. Both domains are wired into the report job, which opens, updates and closes one issue per domain label, and into the domain list that the workflow test and the operations guide keep in step. Neither job continues on error, so a red domain is a red domain. The same test file now reads workflow files with an explicit UTF-8 encoding and compares the published result path and the interpreter passed to its `uv` shim as POSIX paths, which the Windows suite needs.
On a manual dispatch GitHub can run the workflow from a selected branch, and checkout follows that ref. The liveness scan then read that branch's schedule triggers and compared them with scheduled runs, which GitHub only produces from the default branch. A schedule that exists only on the branch was reported dead, and one removed on the branch was skipped. The legacy liveness job pinned develop for this reason; the domain now does the same, and a test pins it.
1c6dfca to
e488ae4
Compare
There was a problem hiding this comment.
oracle-verdict: success for e488ae4c0f588b873841438e0f4acd1dc08f2659
This result supersedes those for earlier heads; their open review threads still apply.
Tier: low-medium (data-only or small single-area change).
Decision: SHIP, carried from head 1c6dfcaa72553d59bed1de324efab79ccd246f95.
The patch is unchanged since head 1c6dfcaa72553d59bed1de324efab79ccd246f95 was reviewed, so its decision is carried to this head without running a reviewer. Its open review threads still apply.
This branch was successfully deployed
1 participant
What
Adds two domains to the nightly T3 workflow (
nightly-v2.yml):t3:windowsruns the fast unit tier onwindows-latestwith Python 3.12, with nocontinue-on-error. It runspytest tests -m "fast and not (slow or stress or resource_heavy or live_integration)"with the same marker expression as the legacy nightly, without the coverage gate the Linux job already enforces. The report is kept as thet3-windowsartifact.t3:livenessrunsscripts/scheduled_workflow_liveness.pywithactions: readandcontents: read, and keeps its log as thet3-livenessartifact.Both are wired into the
reportjob (needs and domain loop), so each opens, updates and closes onet3:<domain>issue like the other domains, and into the domain list in the workflow test anddocs/operations/nightly-t3.md.Why
In
nightly-v2.ymlthe Windows matrix legs only build the wheel, install it and run the bundled generators. The roughly 36,000-test Windows run exists only in the legacynightly.yml, which also holds the only liveness check. Retiring the legacy workflow would end both without a decision, so the new nightly gets them.Sizing
The Windows 3.12 "Run fast tests" step took 38.13 minutes in run 37776283388 (job 39.37 minutes). The 50 minute timeout is that plus a margin above 25%, and well under the 360 minute default: the Windows 3.11 job in the same run hung at pytest-xdist teardown and ran to the default limit. A test pins the margin.
Also in this change
tests/unit/workflows/test_nightly_t3_workflow.pynow reads files with an explicit UTF-8 encoding, compares the published perf result path as aPathinstead of a string, and passes the interpreter to itsuvshim as a quoted POSIX path. These were Windows failures in the same file.Verification
The workflow test file passes on macOS (49 tests). The new Windows domain can only be proven by a Windows run of the workflow after merge; until then it is untested on a runner.