From ffbfad176a3429d4d9657332c969389d90dca8dc Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:02:35 +0000 Subject: [PATCH 1/4] Stop packaging hadoop-client and avro-ipc in plugin jars Image scanning of gcr.io/cdapio/cdap reports jline 3.9.0 and jetty-http 9.4.51.v20230217 inside database-plugins-*.jar. dependency:tree shows both come from hadoop-client 3.3.6 (compile scope) -> hadoop-yarn-client 3.3.6 (jline 3.9.0, websocket-client -> jetty-client -> jetty-http/-util/-io 9.4.51). The main code only uses org.apache.hadoop.{conf,io,util,mapreduce,mapred.lib.db} classes, which come from hadoop-common / hadoop-mapreduce-client-core that CDAP provides at runtime, so hadoop-client is now provided. This also stops packaging the rest of the Hadoop client tree (httpclient, commons-io 2.8.0, commons-compress 1.21, reload4j, kerby, curator, jersey-client, jackson jaxrs 2.12.7, ...) in the database-plugins jar. The root-managed avro-mapred 1.11.4 pulled avro-ipc and avro-ipc-jetty, which brought velocity-engine-core 2.3 (with commons-io 2.8.0) into the format-avro jar (and through it format-parquet / google-cloud). No plugin uses Avro IPC, so both are excluded. Versions: - hadoop-client 3.3.6: compile -> provided (database-plugins/pom.xml) - avro-ipc, avro-ipc-jetty 1.11.4: excluded from avro-mapred (root dependencyManagement) Bugs: Plugins side done for jline b/578444907, b/578489244, b/578492108, b/578644778, b/578663138 and jetty-http b/578515513. Risk: low. No main-code change; Hadoop classes are supplied by CDAP. Downstream projects that relied on database-plugins pulling hadoop-client transitively at compile scope must declare it themselves (none in this repo). Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl database-plugins,format-avro,format-parquet -am: BUILD SUCCESS; hydrator-common, format-common, format-avro, format-parquet and database-plugins unit tests pass (database-plugins: 35 run, 1 skipped: DBConnectorTest assumes live-DB system properties). dependency:tree -Dincludes=org.jline,org.eclipse.jetty,org.apache.avro:avro-ipc*,org.apache.velocity shows jline/jetty only at provided/test scope and no avro-ipc or velocity. unzip -l database-plugins jar: lib/ now only has hydrator-common, guava, opentracing-* and common-0.1.4 (no jline, jetty, hadoop-*); format-avro jar has no avro-ipc / velocity. DB source/sink against real MySQL / Postgres not run (needs a database). --- database-plugins/pom.xml | 2 ++ pom.xml | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/database-plugins/pom.xml b/database-plugins/pom.xml index 90524b353..d4b9fcf89 100644 --- a/database-plugins/pom.xml +++ b/database-plugins/pom.xml @@ -35,6 +35,8 @@ org.apache.hadoop hadoop-client + + provided io.cdap.cdap diff --git a/pom.xml b/pom.xml index 594766c6c..12a3bfc27 100644 --- a/pom.xml +++ b/pom.xml @@ -624,6 +624,15 @@ avro-mapred ${avro.version} + + + org.apache.avro + avro-ipc + + + org.apache.avro + avro-ipc-jetty + org.mortbay.jetty jetty From bd562808683b9076f41857e48be726ce12878b62 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:23:06 +0000 Subject: [PATCH 2/4] Bump log4j2, bcprov, commons-io, json-smart and iq80 snappy Image scanning of gcr.io/cdapio/cdap reports CVEs in jars packaged by hydrator-plugins: log4j-core 2.20.0 / log4j-api 2.21.1 (format-xls, core-plugins via poi), bcprov-jdk15to18 1.84 (core-plugins), commons-io 2.5 (format-delimited), json-smart 2.2.1 (transform-plugins via json-path 2.2.0) and iq80 snappy 0.2 (format-orc via orc-core 1.1.0). The root POM now imports log4j-bom and manages commons-io, json-smart and iq80 snappy, so modules no longer pin their own versions. json-path stays on 2.2.0 (2.9 would pull json-smart 2.5.x). Versions: - log4j2 (log4j-core, log4j-api) 2.20.0 / 2.21.1 -> 2.25.5 (root log4j2.version, log4j-bom import; format-xls log4j-core.version removed) - bcprov-jdk15to18 1.84 -> 1.85 (core-plugins) - commons-io 2.5 / 2.15.0 -> 2.16.1 (root commons-io.version + managed entry; format-delimited property and core-plugins / format-xls pins removed) - json-smart 2.2.1 -> 2.4.10 (root json-smart.version, managed) - iq80 snappy 0.2 -> 0.5 (root iq80-snappy.version, managed) Bugs: Plugins side done for log4j-core b/578431300, b/578491647, b/578514361, b/578514875. iq80 snappy is rows only (b/578567520 stays open through the Platform leveldb exception). Risk: low-medium. Same-major bumps. log4j 2.25.x and commons-io 2.16.1 still run on Java 8. commons-compress (1.26.0 in core-plugins / format-xls) is not touched in this PR. Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl core-plugins,format-xls,format-delimited,transform-plugins,format-orc -am: BUILD SUCCESS; core-plugins 328 tests (2 skipped), transform-plugins 100 (1 skipped), format-xls / format-delimited / format-orc and dependencies all pass. dependency:tree -Dincludes=org.apache.logging.log4j,org.bouncycastle,commons-io,net.minidev:json-smart,org.iq80.snappy shows only log4j 2.25.5, bcprov-jdk15to18 1.85, commons-io 2.16.1, json-smart 2.4.10 and snappy 0.5 at compile scope (smbj's bcprov-jdk15on 1.70 is runtime scope and is not embedded). unzip -l of the built jars: core-plugins lib/ has bcprov-jdk15to18-1.85, commons-io-2.16.1, log4j-api-2.25.5; format-xls has log4j-core/-api 2.25.5, commons-io-2.16.1; format-delimited commons-io-2.16.1; transform-plugins json-smart-2.4.10; format-orc snappy-0.5. Excel / ORC / SFTP pipelines in a sandbox not run. --- core-plugins/pom.xml | 3 +-- format-delimited/pom.xml | 2 -- format-xls/pom.xml | 3 --- pom.xml | 28 ++++++++++++++++++++++++++++ 4 files changed, 29 insertions(+), 7 deletions(-) diff --git a/core-plugins/pom.xml b/core-plugins/pom.xml index 74e7d5c49..8ffed57ca 100644 --- a/core-plugins/pom.xml +++ b/core-plugins/pom.xml @@ -136,7 +136,7 @@ org.bouncycastle bcprov-jdk15to18 - 1.84 + 1.85 @@ -196,7 +196,6 @@ commons-io commons-io - 2.15.0 org.apache.commons diff --git a/format-delimited/pom.xml b/format-delimited/pom.xml index c17c45571..b35a556b8 100644 --- a/format-delimited/pom.xml +++ b/format-delimited/pom.xml @@ -30,7 +30,6 @@ jar - 2.5 3.1 @@ -78,7 +77,6 @@ commons-io commons-io - ${commons-io.version} diff --git a/format-xls/pom.xml b/format-xls/pom.xml index c3ef74b6f..8d40cf018 100644 --- a/format-xls/pom.xml +++ b/format-xls/pom.xml @@ -27,7 +27,6 @@ jar 5.2.5 - 2.20.0 @@ -45,7 +44,6 @@ org.apache.logging.log4j log4j-core compile - ${log4j-core.version} io.cdap.cdap @@ -71,7 +69,6 @@ commons-io commons-io - 2.15.0 org.apache.commons diff --git a/pom.xml b/pom.xml index 12a3bfc27..f7dc69b85 100644 --- a/pom.xml +++ b/pom.xml @@ -88,6 +88,7 @@ 2.0.2.2 1.10 3.1 + 2.16.1 2.6 3.5 3.2.2 @@ -101,6 +102,7 @@ 13.0.1 3.3.6 2.2.4 + 0.5 1.4.1 4.13.1 2.24.0 @@ -111,6 +113,8 @@ 3.4.5 2.2.0 20160212 + 2.4.10 + 2.25.5 4.1.75.Final 1.3.0 3.3.2 @@ -133,6 +137,30 @@ + + org.apache.logging.log4j + log4j-bom + ${log4j2.version} + pom + import + + + commons-io + commons-io + ${commons-io.version} + + + + net.minidev + json-smart + ${json-smart.version} + + + + org.iq80.snappy + snappy + ${iq80-snappy.version} + org.slf4j slf4j-api From f3d6233a2a68f52978dbf8cceba26119f56b861d Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:25:05 +0000 Subject: [PATCH 3/4] Bump org.json to 20260719 Image scanning of gcr.io/cdapio/cdap reports org.json:json 20160212 in transform-plugins-*.jar (3 CVEs). transform-plugins is the only module that uses ${json.version}; only XMLToJSON uses org.json (JSONObject, XML.toJSONObject). Kept separate from the other minor bumps because newer org.json changes behavior: numbers are parsed as BigDecimal / BigInteger where needed, and XML.toJSONObject keeps strings with leading zeros and some other values that the old version coerced. Versions: - org.json:json 20160212 -> 20260719 (root json.version) Bugs: none closed alone in hydrator-plugins (the 3 org.json bugs close once WR-3b and GC-3b also land). Risk: medium. The XMLToJSON output can differ for numeric-looking values (for example leading zeros and very large / high-precision numbers). Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl transform-plugins -am: BUILD SUCCESS; transform-plugins 100 tests pass (1 skipped, ValueMapperTest, as on the HP-2a run), including XMLToJSONConverterTest (4), XMLParserTest (8), XMLMultiParserTest (2), JSONParserTest (8), JSONFormatterTest (2). unzip -l transform-plugins jar: lib/json-20260719.jar (no json-20160212). A sandbox pipeline with XMLToJSON was not run. --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index f7dc69b85..1dbb14f79 100644 --- a/pom.xml +++ b/pom.xml @@ -112,7 +112,7 @@ 4.0.3 3.4.5 2.2.0 - 20160212 + 20260719 2.4.10 2.25.5 4.1.75.Final From d6115dec0cc2ff2c6f71a9d5a5b3c71290d03ab5 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:44:23 +0000 Subject: [PATCH 4/4] Import jackson-bom 2.18.12 Image scanning of gcr.io/cdapio/cdap reports jackson-databind 2.13.4.2 and jackson-core 2.13.4 / 2.14.3 in format-avro, format-parquet (also nested in google-cloud through format-avro) and spark-plugins jars. The root POM now imports com.fasterxml.jackson:jackson-bom so every jackson module (core, databind, annotations, module-scala for the Spark tests) resolves to one version; jackson.databind.version now points at the new jackson.version property, so existing module references keep working. Versions: - jackson (core, databind, annotations, modules) 2.13.4.2 / 2.14.3 -> 2.18.12 (root jackson.version + jackson-bom import) Bugs: none closed alone; the jackson rows still have a copy shaded in parquet-jackson 1.12.0 (HP-4) and in Platform / other plugins. Risk: medium. 2.15 added stream read limits (max string length 20M, nesting depth 1000, number length 1000) and 2.18 changed some property introspection. 2.18.x still runs on Java 8. The repo has no maven-shade-plugin usage, so the shade >= 3.5 rule does not apply. Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify (whole reactor: 17 modules): BUILD SUCCESS; all unit tests pass (core-plugins 328 / 2 skipped, transform-plugins 100 / 1 skipped, database-plugins 35 / 1 skipped, spark-plugins, http-plugins, solrsearch-plugins, format-* modules). dependency:tree -Dincludes=com.fasterxml.jackson.core,com.fasterxml.jackson.module,com.fasterxml.jackson.dataformat shows only 2.18.12 at compile scope. unzip -l: format-avro, format-parquet and spark-plugins lib/ contain jackson-*-2.18.12 only (format-parquet still has parquet-jackson-1.12.0, fixed in HP-4). The submodules (google-cloud, kafka, azure, ...) are separate repos and are not built here (-DcloudBuild). --- pom.xml | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 1dbb14f79..165459a08 100644 --- a/pom.xml +++ b/pom.xml @@ -118,7 +118,8 @@ 4.1.75.Final 1.3.0 3.3.2 - 2.13.4.2 + 2.18.12 + ${jackson.version} ${project.basedir}/src/test/java/ @@ -137,6 +138,13 @@ + + com.fasterxml.jackson + jackson-bom + ${jackson.version} + pom + import + org.apache.logging.log4j log4j-bom