From cfdfe35ee90acaa50c7bf3aa2a1ab640740f5485 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:11:34 +0000 Subject: [PATCH 1/3] Move netty-all, netty-http and kafka_2.12 to test scope kafka-plugins-client declared netty-all, netty-http and the kafka_2.12 broker at compile scope, so the plugin jar packaged about 30 Netty 4.1.75 / 4.1.48 modules, kafka_2.12 2.6.0 and ZooKeeper 3.5.8 under lib/. Image scanning of gcr.io/cdapio/cdap reports the Netty, kafka_2.12 and ZooKeeper CVEs against /opt/cdap/master/artifacts/kafka-plugins-client-*.jar/lib/. Only the embedded-broker unit tests use these jars: every kafka.*, scala.*, io.netty.*, io.cdap.http.* and org.apache.zookeeper.* import is under src/test. Main code uses only org.apache.kafka.clients/common from kafka-clients. The root POM already declares netty-all and netty-http as test scope; the client module overrode that. Versions: - io.netty:netty-all 4.1.75.Final: compile -> test (kafka-plugins-client) - io.cdap.http:netty-http 1.3.0: compile -> test (kafka-plugins-client) - org.apache.kafka:kafka_2.12 2.6.0: compile -> test (kafka-plugins-client), which also drops zookeeper 3.5.8, scala-library/-reflect 2.12.11, jackson-module-scala and metrics-core from the plugin jar Bugs: b/578427766, b/578434052, b/578497139 (kafka_2.12); Plugins side done for the kafka-plugins Netty bugs and zookeeper b/578633351. Risk: low. No main-code dependency on the removed jars. Scala comes from the Spark runtime that already provides spark-streaming at runtime. Testing: JDK 8, Checkstyle on (0 violations). mvn -B clean install (all modules): BUILD SUCCESS, 13 tests, 0 failures (KafkaBatchSourceTest, KafkaSinkAndAlertsPublisherTest, KafkaStreamingSource*Test, KafkaStreamingSourceUtilTest). unzip -l kafka-plugins-client-3.5.0-SNAPSHOT.jar: lib/ goes from 74 to 26 jars; no netty-*, kafka_2.12-*, zookeeper-* or scala-* left; nothing added. mvn dependency:tree -Dincludes=io.netty,org.apache.kafka, org.apache.zookeeper,io.cdap.http: kafka_2.12, netty-all and netty-http are test scope; the only compile-scope match is kafka-clients 2.6.0. --- kafka-plugins-client/pom.xml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kafka-plugins-client/pom.xml b/kafka-plugins-client/pom.xml index 650bfc9..312be9a 100644 --- a/kafka-plugins-client/pom.xml +++ b/kafka-plugins-client/pom.xml @@ -65,6 +65,8 @@ org.apache.kafka kafka_2.12 ${kafka.version} + + test org.slf4j @@ -203,11 +205,13 @@ io.netty netty-all ${netty.version} + test io.cdap.http netty-http ${netty-http.version} + test com.fasterxml.jackson.core From 066a5cfcfba53f0bff4e1bd53cadd2a14a34a798 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:23:20 +0000 Subject: [PATCH 2/3] Upgrade Jackson to 2.18.12 via jackson-bom The kafka plugin jar packages jackson-databind 2.10.5.1 and jackson-core 2.10.5. Image scanning of gcr.io/cdapio/cdap reports the jackson-databind and jackson-core CVEs against /opt/cdap/master/artifacts/kafka-plugins-client-*.jar/lib/. Import com.fasterxml.jackson:jackson-bom in the root dependencyManagement so every Jackson 2 module (including the ones that come transitively through kafka_2.12 and the CDAP test jars) resolves to one version. Versions: - jackson 2.10.5.1 -> 2.18.12 (root jackson.version, now also the jackson-bom import version; kafka-plugins-client jackson-databind keeps using ${jackson.version}) Bugs: no single-ID list for the kafka-plugins Jackson rows in the plan; this is the Plugins side for the kafka-plugins jackson-databind and jackson-core rows. Risk: low. Jackson 2.x minor upgrade; plugin main code has no direct Jackson imports. jackson-core-asl / jackson-mapper-asl 1.9.13 (Jackson 1, from avro 1.8.2) are untouched here and go away with the avro upgrade. Testing: JDK 8, Checkstyle on (0 violations). mvn -B clean install (all modules): BUILD SUCCESS, 13 tests, 0 failures. mvn dependency:tree -pl kafka-plugins-client -Dincludes= com.fasterxml.jackson.*: every Jackson 2 artifact resolves to 2.18.12; compile scope = jackson-databind/-core/-annotations 2.18.12. Plugin jar lib/ has jackson-databind-2.18.12, jackson-core-2.18.12, jackson-annotations-2.18.12 and no 2.10.x jar. --- pom.xml | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index c7016f1..d48c9a2 100644 --- a/pom.xml +++ b/pom.xml @@ -88,9 +88,22 @@ ${project.basedir} 1.0.9 1.7.25 - 2.10.5.1 + 2.18.12 + + + + + com.fasterxml.jackson + jackson-bom + ${jackson.version} + pom + import + + + + io.cdap.cdap From 75a2b6473365af95e83a31d62f37f72e10790e76 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:47:12 +0000 Subject: [PATCH 3/3] Upgrade avro to 1.11.4 and commons-lang3 to 3.18.0 The kafka plugin jar packages avro 1.8.2 (which also shades Guava 11), commons-compress 1.8.1 and commons-lang3 3.9. Image scanning of gcr.io/cdapio/cdap reports the avro, commons-compress and commons-lang3 CVEs against /opt/cdap/master/artifacts/kafka-plugins-client-*.jar/lib/. avro 1.9 removed the Jackson 1 and Joda-Time APIs. Plugin main code has no direct avro imports (avro reaches it through cdap-formats), and the build compiles and the unit tests pass without code changes. commons-compress 1.26.x calls commons-io 2.12+ APIs (for example CloseShieldInputStream.wrap in the bzip2 codec that avro uses). Without a direct dependency Maven mediated commons-io to 2.4 from the provided hadoop-common, and it was not packaged, so the plugin would hit NoSuchMethodError / NoClassDefFoundError on the avro bzip2/xz/zstd codecs. kafka-plugins-client now declares commons-io 2.16.1. Versions: - avro 1.8.2 -> 1.11.4 (root avro.version) - commons-compress 1.8.1 -> 1.26.2 (root dependencyManagement) - commons-lang3 3.9 -> 3.18.0 (root dependency) - commons-io (not packaged, 2.4 at build time) -> 2.16.1 (kafka-plugins-client, new direct dependency) Bugs: no single-ID list for the kafka-plugins avro / commons-compress rows in the plan; this is the Plugins side for those rows. Also removes the Guava 11 shaded in avro 1.8.2 (Guava bugs b/578439199, b/578506400, b/578653882, b/578733862 stay out of scope; guava-13.0.1 is still packaged). Risk: medium. avro crosses the 1.9 API break (Jackson 1 / Joda removal). jackson-core-asl / jackson-mapper-asl 1.9.13 are no longer packaged. Needs a Kafka source/sink run with the avro format in a sandbox. Testing: JDK 8, Checkstyle on (0 violations). mvn -B clean install (all modules): BUILD SUCCESS, 13 tests, 0 failures. mvn dependency:tree -pl kafka-plugins-client: avro 1.11.4 -> commons-compress 1.26.2, commons-io 2.16.1, commons-lang3 3.18.0 (all compile); no avro 1.8.2 / commons-compress 1.8.1 at compile scope (avro 1.8.2 remains only under provided spark-core avro-mapred). Plugin jar lib/: avro-1.11.4, commons-compress-1.26.2, commons-io-2.16.1, commons-lang3-3.18.0; no jackson-*-asl, xz-1.5 or snappy-java-1.1.1.3. Not run: Kafka avro-format pipeline in a sandbox (P2). --- kafka-plugins-client/pom.xml | 10 ++++++++++ pom.xml | 10 ++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/kafka-plugins-client/pom.xml b/kafka-plugins-client/pom.xml index 312be9a..35b03c2 100644 --- a/kafka-plugins-client/pom.xml +++ b/kafka-plugins-client/pom.xml @@ -61,6 +61,16 @@ + + + commons-io + commons-io + 2.16.1 + org.apache.kafka kafka_2.12 diff --git a/pom.xml b/pom.xml index d48c9a2..c0e26ed 100644 --- a/pom.xml +++ b/pom.xml @@ -84,7 +84,7 @@ 2.6.0 4.1.75.Final 1.3.0 - 1.8.2 + 1.11.4 ${project.basedir} 1.0.9 1.7.25 @@ -101,6 +101,12 @@ pom import + + + org.apache.commons + commons-compress + 1.26.2 + @@ -275,7 +281,7 @@ org.apache.commons commons-lang3 - 3.9 + 3.18.0 com.google.guava