Repository navigation
CASSJAVA-144: Move ASF CI from ci-cassandra Jenkins to GH Action - #2104
SiyaoIsHiding wants to merge 13 commits into
Conversation
| echo "CCM_UPDATE_PID_DEFAULT_TIMEOUT=120" >> "$GITHUB_ENV" | ||
| # "-" tells bin/cassandra not to use jemalloc, skipping its unbounded `find` over the | ||
| # library dirs, which can exceed the pid timeout on a cold cache under parallel tests. | ||
| echo "CASSANDRA_LIBJEMALLOC=-" >> "$GITHUB_ENV" |
There was a problem hiding this comment.
This line is prolly the only essential difference from the Node.js driver's GHA. AI found it out this is the reason why sometimes the first integration test likes to fail with cassandra pid not found.
It makes sense to me. I think I will port this change to Node.js driver's GHA, too.
There was a problem hiding this comment.
🟡 Changes recommended
The CI report scan splits filenames containing spaces and suppresses read errors, allowing jobs to pass despite TCK failures.
3 open issues
What changed in this PR
Migrates the Java driver’s ASF CI from Jenkins to GitHub Actions, with reusable test-environment setup and test-result reporting.
Changes:
- Adds a Java 17/21 test matrix against Cassandra 3.11, 4.0, 4.1, and 5.0.
- Adds CCM setup, report uploads, and result publication.
- Removes Jenkins infrastructure and switches documentation builds to Java 17.
| File | Description |
|---|---|
Jenkinsfile-asf |
Removes the ASF Jenkins pipeline. |
ci/run-tests.sh |
Removes the Jenkins test runner. |
ci/install-jdk.sh |
Removes the legacy JDK installer. |
ci/create-user.sh |
Removes the container user-setup script. |
.github/workflows/publish-docs-gp-pages.yml |
Uses Java 17 for documentation builds. |
.github/workflows/ci.yml |
Adds matrix testing and test-result reporting. |
.github/actions/setup-ccm/action.yml |
Prepares CCM, JDKs, certificates, Simulacron, and Cassandra. |
Review effort: Balanced
| on: | ||
| push: | ||
| pull_request: | ||
| workflow_dispatch: # allow manual trigger from the Actions page |
There was a problem hiding this comment.
I mean, I guess...
The only thing scheduled builds are going to do for us are pick up variation in the patch version of a given Cassandra major/minor tree. Same is true for ccm, right, and ccm changes even less frequently. I guess I don't mind scheduled builds but it seems like the benefit here would be pretty small.
There was a problem hiding this comment.
Added cron.
As for other branches, I think we can get this PR merged first, and we will see whether it works well, then we can talk about migrating other branches from Jenkins to GHA, too.
| using: composite | ||
| steps: | ||
| # ---- Python for ccm ---- | ||
| - name: Set up Python 3.15.2 |
There was a problem hiding this comment.
This is another diff from the Node.js driver - installing python 3.9 started to fail on default github action runners because the default one is updated to Ubuntu 26.04, and python 3.9 is EOL.
so, the python version is updated to 3.15, and the below
python -m pip install --upgrade pip setuptools
pip install .
I will update the Node.js driver CI on this changes, too.
|
|
||
| - name: Run unit and integration tests | ||
| run: | | ||
| mvn -B -V verify -T 1 \ |
There was a problem hiding this comment.
The old invocation was:
mvn -B -V verify -T 1 -Ptest-jdk-${TEST_JAVA_MAJOR_VERSION} -DtestJavaHome=$(jabba which ${TEST_JAVA_VERSION}) -Dccm.version=${PATCH_SERVER_VERSION} -Dccm.dse=false -Dmaven.test.failure.ignore=true -Dmaven.javadoc.skip=true
It seems we are missing -Ptest-jdk-${TEST_JAVA_MAJOR_VERSION} is that intentional?
There was a problem hiding this comment.
Also, have we explored the option of using more threads? -T 1 is the slowest option (but also the safest).
There was a problem hiding this comment.
The profiles -Ptest-jdk-${TEST_JAVA_MAJOR_VERSION} were removed
Re: -T 1. Our integration tests are already categorized under ParallelizableTests, which will run with 8 threads even if we pass -T 1, and SerialTests and IsolatedTests, which won't run concurrently. So, the tests that can benefit from concurrency already run concurrently.
| id: z11 | ||
| uses: actions/setup-java@v5 | ||
| with: | ||
| distribution: zulu |
There was a problem hiding this comment.
Any specific reason to go with Zulu? The docs CI uses temurin, which seems a more common choice.
There was a problem hiding this comment.
Tried out temurin, it fails to fetch java 8 quite often, see 1, 2
Seems like temurin 8 is fetched from https://github.com/adoptium/temurin8-binaries/releases/download/jdk8u514-b01/OpenJDK8U-jdk_x64_linux_hotspot_8u514b01.tar.gz, and it's flaky
While zulu is fetched from https://cdn.azul.com/zulu/bin/zulu8.98.0.201-ca-jdk8.0.514-linux_x64.tar.gz, and I didn't see an error so far.
There was a problem hiding this comment.
the doc CI uses temurin 17 and it's fine because temurin 17 is cached on ubuntu 26.
And the doc CI was using temurin java 8, it also didn't have a problem in the past because the default github runner was ubuntu 24, and ubuntu 24 also had temurin java 8 in the cache.
| path: artifacts | ||
|
|
||
| - name: Publish test results | ||
| uses: EnricoMi/publish-unit-test-result-action@v2 |
There was a problem hiding this comment.
I double-checked that this action is allow-listed by the ASF, and it is. That said, it would be safer to pin it to a specific commit SHA rather than a tag.
There was a problem hiding this comment.
Also, I wonder if this action will fail when executed from a forked repo, because of the required privileges: checks: write pull-requests: write. I haven't confirmed this though.
If that proves correct, it could be solved by switching off checks and comments and keeping only the summary:
- uses: EnricoMi/publish-unit-test-result-action@v2
with:
files: artifacts/**/TEST-*.xml
comment_mode: off
check_run: false... in which case, we don't need checks: write pull-requests: write.
There was a problem hiding this comment.
I looked into this more and learnt:
That action does mainly three things
- Check run. Including annotatoins.
- Pull request comment.
- Job summary
For a PR from a fork. 1️⃣ and 2️⃣ won't work, but 3️⃣ will work fine.
Node.js driver uses the same config, it looks like this: https://redirect.github.com/apache/cassandra-nodejs-driver/pull/472
The job won't fail and we can still see which runs have test failures. I think it's good enough.
Unless we set up a separate workflow just for 1️⃣ and 2️⃣ , which I find unnecessary, the checks: write, or comment_mode: off, or check_run: false won't do anything, cuz 1️⃣ and 2️⃣ won't work anyway.
I think the best might be removing checks: write and add comment_mode: off and check_run: false just to be even clearer. What do you think?
| name: Unit and Integration Tests | ||
|
|
||
| on: | ||
| push: |
There was a problem hiding this comment.
Suggestion: restrict to pushes to main branches only:
| push: | |
| push: | |
| branches: | |
| - trunk # default branch; adjust if it's named differently | |
| - '[0-9]*.x' # release branches: 3.x, 4.x, 4.5.x, ... |
There was a problem hiding this comment.
If we don't add this restriction, we will have a perk that, anyone with a fork, after they click the button to enable github actions in this fork, will have CI automatically run on their feature branches even without a pull request.
Do you think it's worth it?
| # Saturday 23:00 PST / Sunday 00:00 PDT; cron is in UTC and runs on the default branch only | ||
| - cron: "0 7 * * 0" | ||
|
|
||
| permissions: |
There was a problem hiding this comment.
Suggestion: add a concurrency section to limit the number of concurrent runs, since the matrix is quite large:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}| workflow_dispatch: # allow manual trigger from the Actions page | ||
| schedule: | ||
| # Saturday 23:00 PST / Sunday 00:00 PDT; cron is in UTC and runs on the default branch only | ||
| - cron: "0 7 * * 0" |
There was a problem hiding this comment.
It seems that on Jenkins, a weekly job would run on every release branch. Here, we are losing that (this cron applies only to the main branch, afaik). To recover the ability to run CI weekly on all release branches, we'd need a separate workflow.
There was a problem hiding this comment.
yeah, after this PR gets merged and if everything works well, I'll create subsequent PRs to 3.x and 4.x branch, and they will have their own copies of ci.yaml files, which will have their own crons.
| unit-integration-tests: | ||
| name: Unit and Integration Tests (JDK ${{ matrix.TEST_JAVA_VERSION }}, C* ${{ matrix.SERVER_VERSION }}) | ||
| runs-on: ubuntu-latest | ||
| strategy: |
There was a problem hiding this comment.
Suggestion: add a hard timeout:
timeout-minutes: 120| @@ -0,0 +1,139 @@ | |||
| name: Set up CCM for integration tests | |||
There was a problem hiding this comment.
Missing the ASF license header.
| - name: Print environment | ||
| shell: bash | ||
| run: printenv | sort |
There was a problem hiding this comment.
This can print sensitive data. Do we need it?
| shell: bash | ||
| run: | | ||
| python -m pip install --upgrade pip setuptools | ||
| git clone --depth 1 --single-branch -b cassandra-test https://github.com/apache/cassandra-ccm.git |
There was a problem hiding this comment.
That's a tag, used by cassandra's dtest.
I think either using trunk or pin a commit of the trunk is fine.
This reverts commit 4dcbbae.


Example run:
https://github.com/SiyaoIsHiding/java-driver/actions/runs/40934985232
Largely reusing Node.js driver's GHA. Example user interface on a PR: https://redirect.github.com/apache/cassandra-nodejs-driver/pull/472
Disclose of Generative AI usage.