fix(docker): replace outdated tcnative 1.2.35 with Apache tcnative 1.3.8 by wezell · Pull Request #37399 · dotCMS/core

wezell · 2026-09-04T13:32:17Z

Summary

dotCMS's SSL offloading to openssl has been broken since we moved to Ubuntu 24.04. We haven't noticed because it would fall back to using java's SSL but this has performance implications and it is better to offload it.

Fixes #34067

dotCMS crashed on startup with a JVM SIGSEGV inside libcrypto.so.3 when the Tomcat Native APR library initialized OpenSSL 3.x. Root cause: the Ubuntu 24.04 libtcnative-1 package (1.2.35, Jan 2023) is incompatible with OpenSSL 3.x — the very library version all our images install to offload Tomcat 9's SSL.

Tomcat 9 requires the tcnative 1.x line (2.x is Tomcat 10.1+ only). This PR builds the current Apache release, tcnative 1.3.8, from source (SHA-512 pinned) against the image's own APR 1.7 / OpenSSL 3, and stops installing the distro package.

Changes

File	Change
`docker/java-base/Dockerfile`	Builds tcnative 1.3.8 into `/usr/local/tomcat-native` (built once per java-base release, reused downstream); build deps purged; JDK default `25.0.4+1-ms`
`dotCMS/src/main/docker/original/Dockerfile`	Drops distro `libtcnative-1`; copies tcnative from java-base; sets `LD_LIBRARY_PATH`
`docker/dev-env/Dockerfile`	Copies tcnative from the dotcms image (no build of its own)
`15-detect-fips-and-set-ssl-engine.sh`	Comment updated; FIPS guard kept as defense-in-depth
`.sdkmanrc`	Bumped to canonical SDKMAN id `25.0.4+1-ms`
`maven-job` action, `deployment-phase`, `legacy-release`, `java-base` workflows	Convert `+`→`-` where the SDKMAN id becomes a docker tag (docker tags cannot contain `+`)
`parent/pom.xml`	`runtime.docker.sdkman.java.version` defaults to docker-tag-safe `25.0.4-1-ms` (maven cannot transform the `.sdkmanrc`-loaded property)

Deployment note

The java-base image with tcnative is already pushed as dotcms/java-base:25.0.4-1-ms (built via cicd_manual_build-java-base.yml), so downstream builds resolve immediately.

Validation

just build (./mvnw -DskipTests clean install): BUILD SUCCESS, including the maven-driven docker image build
Resulting image (dotcms/dotcms-test:1.0.0-SNAPSHOT): tcnative 1.3.8 present from java-base, no distro libtcnative-1, LD_LIBRARY_PATH set, Java 25.0.4.1
Loaded through Tomcat 9.0.120's own tomcat-jni.jar (what AprLifecycleListener does at startup):
```
Tomcat JNI loaded: tcnative 1.3.8 / APR 1.7.2 / OpenSSL 3.0.13 30 Jan 2024
```
java-base built end-to-end locally (multi-arch safe: JDK path resolved via SDKMAN, not hardcoded arch); jlink'd JRE loads the library via LD_LIBRARY_PATH
All touched workflow/action YAML parses cleanly; +→- substitution verified for both +-containing and plain versions

The Ubuntu 24.04 libtcnative-1 package (1.2.35) segfaults with OpenSSL 3.x during AprLifecycleListener SSL initialization (SIGSEGV in libcrypto.so.3). Tomcat 9 requires the tcnative 1.x line; build the current Apache release (1.3.8, SHA-512 pinned) from source in the java-base image and reuse it in the runtime and dev-env images instead of the distro package. Also handles SDKMAN ids containing '+' (e.g. 25.0.4+1-ms), which are not valid docker tags: the canonical id stays in .sdkmanrc and '+' is converted to '-' wherever the value becomes a docker tag. Fixes #34067

claude · 2026-09-04T13:33:12Z

Claude finished @wezell's task in 2m 59s —— View job

Code Review

I reviewed the full diff against origin/main. The tcnative-from-source approach is sound, the runtime deps (libapr1, openssl→libssl3) are present in every downstream image, LD_LIBRARY_PATH/java.library.path correctly point at /usr/local/tomcat-native/lib, and the +→- docker-tag conversion is applied consistently (and SDKMAN_JAVA_VERSION is only ever consumed as the FROM dotcms/java-base:<tag> tag in the maven flow, not as a sdk install id, so the conversion is safe there).

New Issues

🟡 Medium: dotCMS/src/main/java/com/dotmarketing/db/DataSourceStrategyProvider.java:101-111 — the implicit-resolution chain reorders Docker Secrets ahead of system-env, and combined with dropping the DOT_DATASOURCE_PROVIDER_STRATEGY_CLASS export from setenv.sh, this is a live behavior change for containers. Old behavior: setenv.sh exported the explicit SystemEnvDataSourceStrategy, so env vars always won and the Docker-Secret branch was effectively dead. New behavior: property unset → chain walks db.properties → Docker Secret (if the secret path exists) → system-env. Any deployment that has a Docker Secret file mounted and relied on DB_* env vars will now silently switch to the Docker Secret source. Blast radius is limited (only when a secret path is actually present), and this is arguably the intended fix, but it's worth a conscious call-out. Assumption: DockerSecretDataSourceStrategy.dockerSecretPathExists() returns true only when the secrets path is mounted. What to verify: no shipped/production compose or helm config mounts a secret path while also configuring the DB via DB_* env vars.

Existing

🟡 Medium: dotCMS/src/main/resources/container/tomcat9/conf/server.xml:133 — SSLProtocol="${CMS_SSL_PROTOCOLS:-TLSv1.3}" drops TLS 1.2 clients from the direct HTTPS connector by default (previously Tomcat default = TLSv1.2+TLSv1.3). Consciously accepted by the author as "secure by default, overridable via CMS_SSL_PROTOCOLS". Non-blocking — flagging only so the TLS-1.3-only default is a deliberate, documented choice for anyone terminating TLS directly at dotCMS (not behind a proxy).

Resolved

✅ dotCMS/src/main/java/com/dotmarketing/db/DataSourceStrategyProvider.java:130-146 — prior P1 ("defaulting strategy class bypasses db.properties/Docker Secrets") is fixed: isDataSourceProviderExplicitlyConfigured() uses a null default, so an unset property correctly walks the full precedence chain instead of short-circuiting to SystemEnvDataSourceStrategy. New DataSourceStrategyProviderTest covers the branches. (Minor nit, non-blocking: the "com.dotmarketing.db.SystemEnvDataSourceStrategy" default in getCustomDataSourceProvider() is now only read on the explicit branch where the property is guaranteed set, so it's effectively dead — harmless but slightly misleading.)
✅ docker/java-base/Dockerfile:61 — tarball 404 concern addressed with the downloads.apache.org → archive.apache.org fallback, SHA-512 pin still enforced on whichever URL succeeds.

No blocking issues. The two medium items are behavior-change call-outs, both already discussed/accepted in the PR threads.

Read changed files
Analyze Docker/tcnative changes
Analyze DataSourceStrategyProvider changes
Analyze TLS/server.xml + workflow changes
Post review
· branch issue-34067-fix-tcnative-apr-openssl3

downloads.apache.org only hosts the current release; once tcnative 1.3.8 is superseded the URL would 404 and break java-base builds. Fall back to archive.apache.org, which retains all releases. Checksum still enforced.

dotCMS-Machine-User

✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.

_{approved automatically by dotbot}

github-actions · 2026-09-04T14:25:27Z

🐳 PR Docker test image

Latest build for commit 5168177 pushed to dotcms/dotcms-test:

docker pull dotcms/dotcms-test:pr-37399-issue-34067-fix-tcnative-apr-openssl3
docker pull dotcms/dotcms-test:pr-37399-issue-34067-fix-tcnative-apr-openssl3_5168177

…e library path - DataSourceStrategyProvider: default DATASOURCE_PROVIDER_STRATEGY_CLASS to SystemEnvDataSourceStrategy so DB_* environment variables work out of the box without setting DOT_DATASOURCE_PROVIDER_STRATEGY_CLASS - setenv.sh: prepend /usr/local/tomcat-native/lib to java.library.path. -Djava.library.path overrides LD_LIBRARY_PATH, so the previously hardcoded multiarch path prevented Tomcat's AprLifecycleListener from finding libtcnative-1, silently falling back to JSSE on 8443

…_SSL_PROTOCOLS Ship with TLSv1.3 only (SSLProtocol="${CMS_SSL_PROTOCOLS:-TLSv1.3}"). TLS 1.2 with modern AEAD ciphers remains available by setting CMS_SSL_PROTOCOLS=TLSv1.2+TLSv1.3 (or all) for deployments with legacy TLS 1.2 clients. Validated live: TLS 1.2 rejected, TLS 1.3 accepted, server starts cleanly with the restriction in place.

…escape hatch Review follow-up: the empty-value escape hatch (which restores the legacy db.properties / Docker Secrets resolution order) was undocumented and, worse, unreachable in Docker because setenv.sh force-exported DOT_DATASOURCE_PROVIDER_STRATEGY_CLASS, overriding any operator-provided value. - setenv.sh: drop the forced export; the code default in DataSourceStrategyProvider already covers Dockerized environments - javadoc: document all three resolution modes (default SystemEnv, explicit class override, empty-value escape hatch) and when to use each - add DataSourceStrategyProviderTest covering the default, the explicit override, and the empty-value escape hatch (3 tests)

…stemEnv default Previously, setting DATASOURCE_PROVIDER_STRATEGY_CLASS to default to SystemEnvDataSourceStrategy routed all unconfigured lookups directly to the custom-strategy branch, bypassing existsDBPropertiesFile() and dockerSecretPathExists(). Now DataSourceStrategyProvider preserves the full resolution precedence: 1. Explicit custom class (if configured and != SystemEnvDataSourceStrategy) 2. db.properties (if file exists) 3. Docker Secrets (if secret path exists) 4. SystemEnvDataSourceStrategy (the default when no other config present) 5. TomcatDataSourceStrategy (context.xml fallback on failure) Added unit tests covering all resolution branches (6 tests).

dotCMS-Machine-User

✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.

_{approved automatically by dotbot}

The integration test harness sets DATASOURCE_PROVIDER_STRATEGY_CLASS to SystemEnvDataSourceStrategy explicitly and expects it to run directly via the custom-strategy branch. The previous change treated an explicit SystemEnv as the implicit default and re-routed it through the precedence chain, causing the harness to fall through to the JNDI fallback (NoInitialContextException) and fail MainSuite 2a. Now an explicitly configured value (including an explicit SystemEnvDataSourceStrategy) is honored directly; only the implicit default (property unset) walks the precedence chain (db.properties -> Docker Secrets -> SystemEnv -> context.xml fallback). Extracted isDataSourceProviderExplicitlyConfigured() (VisibleForTesting) and reworked the unit tests to drive the resolution deterministically (7 tests).

dotCMS-Machine-User

✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.

_{approved automatically by dotbot}

github-actions · 2026-09-04T19:54:01Z

dotbot code review:

Reviewer: meta/muse-spark-1.3 (medium)
Overall: patch is correct
New findings this run: 0
Prior unresolved dotbot findings still relevant: 0
Active findings total: 0

Patch replaces distro libtcnative with source-built 1.3.8 and fixes datasource precedence with tests. No P0/P1 bug with concrete in-repo evidence remains; open points are intentional trade-offs.

Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads.

_{reviewed by dotbot · meta/muse-spark-1.3 · medium}

github-actions · 2026-09-04T20:00:55Z

+
+                Logger.info(DataSourceStrategyProvider.class,
+                        "Datasource loaded using custom class " + providerClassName);
+            } else {


🔴 [P1] DataSourceStrategyProvider.java:96 shipped commented-out db.properties now short-circuits env-var datasource

Current code:

if (getDBPropertiesInstance() .existsDBPropertiesFile()) { defaultDataSource = getDBPropertiesInstance() .apply();

Problem: dotCMS/src/main/resources/db.properties ships in the WAR's WEB-INF/classes but is entirely commented out ("This property file is no longer used"). existsDBPropertiesFile() is therefore true in every stock deployment, while HikariConfig(propertiesFile.getPath()) in DBPropertiesDataSourceStrategy.getHikariConfig() fails (no jdbcUrl). Previously setenv.sh force-exported DOT_DATASOURCE_PROVIDER_STRATEGY_CLASS=SystemEnvDataSourceStrategy, so this branch was unreachable in Docker; this patch removes that export and makes the chain reachable. The exception is swallowed in get(), defaultDataSource stays null, and the finally falls back to TomcatDataSourceStrategy, whose JNDI lookup is undefined in the shipped container/tomcat9/conf/context.xml (no <Resource>), so it throws DotRuntimeException and DB init fails for env-var-configured containers. Separately, reordering Docker Secrets ahead of DB_* env vars silently changes resolution for deployments that mount /run/secrets/ while configuring via env vars.

Fix:

} else if (getDBPropertiesInstance().hasActiveJdbcUrl()) { defaultDataSource = getDBPropertiesInstance().apply();

Or stop shipping the vestigial db.properties so SystemEnvDataSourceStrategy resolves for the stock image.

github-actions · 2026-09-04T20:00:55Z

    String getCustomDataSourceProvider() {
        return Config
-                .getStringProperty("DATASOURCE_PROVIDER_STRATEGY_CLASS", null);
+                .getStringProperty("DATASOURCE_PROVIDER_STRATEGY_CLASS", "com.dotmarketing.db.SystemEnvDataSourceStrategy");


🟡 [P2] DataSourceStrategyProvider.java:133 dead default in getCustomDataSourceProvider

Current code:

return Config .getStringProperty("DATASOURCE_PROVIDER_STRATEGY_CLASS", "com.dotmarketing.db.SystemEnvDataSourceStrategy");

Problem: This default is only read on the hasExplicitCustomProvider branch, where the property is guaranteed set — it is dead and misleading.

Fix:

return Config.getStringProperty("DATASOURCE_PROVIDER_STRATEGY_CLASS", null);

If the explicit-config check is ever simplified, this default would silently resurrect the precedence bypass.

github-actions · 2026-09-04T20:00:55Z

        useSendfile="${CMS_USE_SENDFILE:-false}"
        maxHttpHeaderSize="${CMS_MAX_HTTP_HEADER_SIZE:-16384}"
        SSLEnabled="${CMS_SSL_ENABLED:-true}"
+        SSLProtocol="${CMS_SSL_PROTOCOLS:-TLSv1.3}"


🟡 [P2] server.xml:133 TLSv1.3-only default drops TLS 1.2 clients on the direct connector

Current code:

SSLProtocol="${CMS_SSL_PROTOCOLS:-TLSv1.3}"

Problem: Previously Tomcat's default (TLSv1.2+TLSv1.3) applied; now TLS 1.2-only clients are rejected unless CMS_SSL_PROTOCOLS is set — a silent breaking default for deployments terminating TLS directly at dotCMS.

Fix:

SSLProtocol="${CMS_SSL_PROTOCOLS:-TLSv1.2+TLSv1.3}"

Assumption: the hardening is deliberate per the comment. What to verify: no supported clients/proxies negotiate only TLS 1.2 against port 8443.

github-actions · 2026-09-04T20:00:59Z

dotbot code review:

Reviewer: ~z-ai/glm-latest (medium)
Overall: patch is incorrect
New findings this run: 3
Prior unresolved dotbot findings still relevant: 0
Active findings total: 3

The tcnative 1.3.8 build, LD_LIBRARY_PATH plumbing, and docker-tag '+'→'-' handling are sound, but removing the setenv.sh export combined with the new implicit chain makes the shipped, all-commented db.properties take precedence over DB_* env vars in every stock deployment; Hikari fails on it and the JNDI fallback (no Resource in context.xml) throws, breaking the default datasource path.

Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads.

_{reviewed by dotbot · ~z-ai/glm-latest · medium}

wezell requested a review from a team as a code owner September 4, 2026 13:32

github-project-automation Bot added this to dotCMS - Product Planning Sep 4, 2026

github-actions Bot mentioned this pull request Sep 4, 2026

JVM crash on startup due to Tomcat Native APR incompatibility with OpenSSL 3.x #34067

Open

github-actions Bot added Area : Backend PR changes Java/Maven backend code Area : CI/CD PR changes GitHub Actions/workflows labels Sep 4, 2026

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

wezell added PR: docker image Build & push a per-PR test image to dotcms/dotcms-test PR : dotbot review Trigger dotbot AI code review on this PR labels Sep 4, 2026

github-actions Bot reviewed Sep 4, 2026

View reviewed changes

Comment thread docker/java-base/Dockerfile

Comment thread parent/pom.xml

fix(docker): fall back to archive.apache.org for tcnative source

e29b205

downloads.apache.org only hosts the current release; once tcnative 1.3.8 is superseded the URL would 404 and break java-base builds. Fall back to archive.apache.org, which retains all releases. Checksum still enforced.

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

dotCMS-Machine-User previously approved these changes Sep 4, 2026

View reviewed changes

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

wezell dismissed dotCMS-Machine-User’s stale review via fb9d8f4 September 4, 2026 16:13

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

github-actions Bot reviewed Sep 4, 2026

View reviewed changes

Comment thread dotCMS/src/main/java/com/dotmarketing/db/DataSourceStrategyProvider.java

Comment thread dotCMS/src/main/resources/container/tomcat9/conf/server.xml

wezell added 2 commits September 4, 2026 12:34

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

dotCMS-Machine-User previously approved these changes Sep 4, 2026

View reviewed changes

Merge branch 'main' into issue-34067-fix-tcnative-apr-openssl3

d51807e

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

wezell dismissed dotCMS-Machine-User’s stale review via e0d2fc5 September 4, 2026 17:18

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

dotCMS-Machine-User approved these changes Sep 4, 2026

View reviewed changes

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

Merge branch 'main' into issue-34067-fix-tcnative-apr-openssl3

86ca6ce

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

wezell added the CI : No Build Cache Skip the shared S3 Maven build cache for this PR; build everything from source label Sep 4, 2026

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

Merge branch 'main' into issue-34067-fix-tcnative-apr-openssl3

5168177

github-actions Bot removed the AI: Safe To Rollback label Sep 4, 2026

claude Bot added the AI: Safe To Rollback label Sep 4, 2026

github-actions Bot reviewed Sep 4, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix(docker): replace outdated tcnative 1.2.35 with Apache tcnative 1.3.8 - #37399

fix(docker): replace outdated tcnative 1.2.35 with Apache tcnative 1.3.8#37399
wezell wants to merge 10 commits into
mainfrom
issue-34067-fix-tcnative-apr-openssl3

wezell commented Sep 4, 2026 •

edited

Loading

Uh oh!

claude Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

dotCMS-Machine-User left a comment

Uh oh!

github-actions Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

dotCMS-Machine-User left a comment

Uh oh!

dotCMS-Machine-User left a comment

Uh oh!

github-actions Bot commented Sep 4, 2026

Uh oh!

github-actions Bot Sep 4, 2026

Uh oh!

github-actions Bot Sep 4, 2026

Uh oh!

github-actions Bot Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

wezell commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Deployment note

Validation

Uh oh!

claude Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review

New Issues

Existing

Resolved

Uh oh!

Uh oh!

Uh oh!

dotCMS-Machine-User left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🐳 PR Docker test image

Uh oh!

Uh oh!

Uh oh!

dotCMS-Machine-User left a comment

Choose a reason for hiding this comment

Uh oh!

dotCMS-Machine-User left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 4, 2026

Uh oh!

github-actions Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wezell commented Sep 4, 2026 •

edited

Loading

claude Bot commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading