ci: fold test_transformers into the sharded nox job - #692
Conversation
Drop the standalone transformers-py job and let test_transformers run alongside every other session under the sharded nox matrix. The Hugging Face model cache moves into the nox job so shards still avoid re-downloading models across runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 057536c491
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".
| mise exec python@${{ matrix.python-version }} -- python ./py/scripts/nox-matrix.py ${{ matrix.shard }} 6 \ | ||
| --exclude-static-checks |
There was a problem hiding this comment.
When the Hugging Face cache is cold—most notably on the first Windows run—the sharded matrix starts every Python/OS combination concurrently, so none can restore models saved by an earlier Python version. Checking checks.yaml and the parametrized test_transformers session in py/noxfile.py, up to eight jobs per OS can therefore fetch the same five models simultaneously, recreating the download stampede that the removed max-parallel: 1 explicitly prevented and risking Hugging Face throttling or CI timeouts; retain a serialized cache-warming/dedicated job before including these sessions in the shards.
AGENTS.md reference: AGENTS.md:L17-L21
Useful? React with 👍 / 👎.
2 participants
Summary
transformers-pyjob;test_transformersnow runs alongside every other session under the shardednoxmatrix.noxjob so shards still avoid re-downloading models across runs.checks-passedto no longer gate ontransformers-py.Test plan
test_transformers(...)lands in some shard and passes on Linux.session.skipforsys.platform == "win32"innoxfile.py:test_transformers.🤖 Generated with Claude Code
Created by Abhijeet Prasad (@AbhiPrasad)
Slack thread