Repository navigation
Bump jgit, gson, log4j2 and commons-lang3; drop commons-io override - #16307
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates several dependency versions in the parent pom.xml (including Gson, Jetty, Netty, Logback, and HttpClient) and introduces BOM imports for Netty and Jetty to centralize dependency management. Feedback on the changes suggests further simplifying the submodule POM files (such as cdap-kubernetes/pom.xml, cdap-runtime-ext-emr/pom.xml, and cdap-standalone/pom.xml) by removing redundant tags for dependencies like Logback and Apache HttpComponents, since their versions are already managed by the parent POM's .
| <dependency> | ||
| <groupId>ch.qos.logback</groupId> | ||
| <artifactId>logback-classic</artifactId> | ||
| <version>1.2.8</version> | ||
| <version>${logback.version}</version> | ||
| <scope>runtime</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>ch.qos.logback</groupId> | ||
| <artifactId>logback-core</artifactId> | ||
| <version>1.2.8</version> | ||
| <version>${logback.version}</version> | ||
| <scope>runtime</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Since logback-classic and logback-core are managed in the parent pom.xml's <dependencyManagement>, specifying their versions here is redundant. Removing the <version> tags simplifies the configuration and ensures that version management remains centralized in the parent POM.
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<scope>runtime</scope>
</dependency>
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpclient</artifactId> | ||
| <version>4.5.2</version> | ||
| <version>${httpclient.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpcore</artifactId> | ||
| <version>4.4.4</version> | ||
| <version>${httpcore.version}</version> | ||
| <scope>compile</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Since httpclient and httpcore are now managed in the parent pom.xml's <dependencyManagement>, specifying their versions here is redundant. Removing the <version> tags simplifies the configuration and ensures that version management remains centralized in the parent POM.
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<scope>compile</scope>
</dependency>
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpclient</artifactId> | ||
| <version>4.5.2</version> | ||
| <version>${httpclient.version}</version> | ||
| </dependency> |
There was a problem hiding this comment.
Since httpclient is now managed in the parent pom.xml's <dependencyManagement>, specifying its version here is redundant. Removing the <version> tag simplifies the configuration and ensures that version management remains centralized in the parent POM.
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> | ||
| <artifactId>httpcore</artifactId> | ||
| <version>4.4.4</version> | ||
| <version>${httpcore.version}</version> | ||
| <scope>compile</scope> | ||
| </dependency> |
There was a problem hiding this comment.
Since httpcore is now managed in the parent pom.xml's <dependencyManagement>, specifying its version here is redundant. Removing the <version> tag simplifies the configuration and ensures that version management remains centralized in the parent POM.
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<scope>compile</scope>
</dependency>
cdap-data-streams3_2.12 was the only system-artifact module still on maven-bundle-plugin 3.3.0. cdap-data-pipeline3_2.12, cdap-etl-batch and cdap-program-report already use 3.5.0. The bnd in 3.3.0 can't parse the root-level module-info.class shipped in gson 2.8.9 and jackson-annotations 2.18.12 (the CVE bumps in this stack). Packaging fails with "Invalid class file module-info.class (java.lang.ArrayIndexOutOfBoundsException: 19)", and cdap-master -Pdist can't produce the data-streams system artifact. 3.5.0 handles it. Checked locally (JDK 8): - On develop, the data-streams bundle built with 3.3.0 and with 3.5.0 is the same: same 28 embedded jars, same Export-Package and Import-Package, same Require-Capability. - On this stack, packaging fails with 3.3.0 and succeeds with 3.5.0. The only bundle differences from develop are the intended version bumps (gson 2.8.9, logback 1.2.13). maven-bundle-plugin 5.1.9 was also tried. It builds, but changes what gets embedded: 11 to 17 extra jars per bundle, including cdap-api-common, cdap-runtime-spi, scala-library, jetty 9.4.51 and jackson 2.15.1. So it was not used here.
Root pom.xml: - jgit 5.9.0.202009080501-r -> 5.13.5.202508271544-r (stays on 5.x; 6.x needs Java 11). 5.13.4.202507202350-r fixes the same CVEs but was built on JDK 17 without --release 8: it calls the Java 9+ ByteBuffer.mark() overload and fails on Java 8 with NoSuchMethodError (all RepositoryManagerTest cases fail). 5.13.5 is the 5.13.4 fix release for this. - gson.version 2.3.1 -> 2.8.9. - log4j2.slf4j.version (log4j-to-slf4j / log4j-api) 2.20.0 -> 2.25.5. - commons-lang3 3.12.0 -> 3.18.0 (must precede commons-configuration2 2.15.0 in CL-07). cdap-master/pom.xml: remove the commons-io 2.12.0 override in the k8s profile so master/lib uses the managed commons-io 2.15.1. The quartz 2.3.2 bump needs a code change, so it is in the next commit (vuln/cl-09b-code) together with that change; every commit builds. Why: CVE fixes from image scanning of gcr.io/cdapio/cdap. Closes jgit b/578567823 (High) and b/578495338 (Medium). Platform side of log4j-api b/578584043 and commons-io b/578659719. Risk: low to medium. gson is used in many modules and is visible to plugins. Testing: JDK 8, Checkstyle on: cdap-source-control (all tests), cdap-proto (all tests), cdap-common LocationsTest, cdap-app-fabric compile and TimeSchedulerTest; mvn dependency:tree for the bumped artifacts.
876c4b9 to
e1b244f
Compare
| <groupId>org.apache.felix</groupId> | ||
| <artifactId>maven-bundle-plugin</artifactId> | ||
| <version>3.3.0</version> | ||
| <version>3.5.0</version> |
There was a problem hiding this comment.
Can the latest 3.x version - 3.5.1 be used instead?
| <junit.version>4.11</junit.version> | ||
| <hamcrest.version>2.2</hamcrest.version> | ||
| <log4j2.slf4j.version>2.20.0</log4j2.slf4j.version> | ||
| <log4j2.slf4j.version>2.25.5</log4j2.slf4j.version> |
There was a problem hiding this comment.
Can 2.26.1 be used instead as it have additional vuln fixes?
There was a problem hiding this comment.
Thanks. 3.5.1 would work too: it embeds the same bndlib 3.5.0, so it handles module-info.class the same way. I kept 3.5.0 because every other module that uses the plugin (data-pipeline, etl-batch, program-report, the plugins archetype) is on 3.5.0, and the bundle comparison and E2E runs were done with it. I can move all of them to 3.5.1 together in a follow-up if you prefer.
|



Bump jgit, gson, log4j2 and commons-lang3; drop commons-io override
Repo: cdapio/cdap
Base: develop (stacked; retarget to develop after the lower PRs merge)
Commit: 876c4b9
Title
Bump jgit, gson, log4j2 and commons-lang3; drop commons-io override
Description
Changes in the root
pom.xml:because 6.x needs Java 11.
gson.version2.3.1 → 2.8.9.log4j2.slf4j.version(log4j-to-slf4j and log4j-api) 2.20.0 → 2.25.5.commons-configuration2 2.15.0 in Pin Hadoop-transitive jettison, nimbus, beanutils, configuration2 and commons-text #16311.
Change in
cdap-master/pom.xml: removes the commons-io 2.12.0 override fromthe
k8sprofile.master/libnow gets the managed commons-io 2.15.1(root
commons.io.versionis already 2.15.1 on develop).Why: these are CVE fixes from image scanning of gcr.io/cdapio/cdap.
Why jgit 5.13.5 and not the planned 5.13.4:
5.13.4.202507202350-rwasbuilt on JDK 17 without
--release 8. It calls the Java 9+ByteBuffer.mark()overload. On JDK 8, every RepositoryManagerTest case (19)and every RemoteSourceControlOperationRunnerTest case (12) fails with
NoSuchMethodError: java.nio.ByteBuffer.mark()Ljava/nio/ByteBuffer;, thrownfrom
RawParseUtils.decodeNoFallback. 5.13.5 is the release that fixes this.With 5.13.5, all source-control tests pass.
Risk: low to medium.
provided. This PR doesnot package them into
master/lib.Testing (JDK 1.8.0_382, Checkstyle on, on this commit alone):
dependency:tree: jgit 5.13.5, gson 2.8.9, log4j-to-slf4j and log4j-api2.25.5, commons-io 2.15.1, commons-lang3 3.18.0.
mvn test, and S1–S5.