From ffbfad176a3429d4d9657332c969389d90dca8dc Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:02:35 +0000 Subject: [PATCH 1/3] Stop packaging hadoop-client and avro-ipc in plugin jars Image scanning of gcr.io/cdapio/cdap reports jline 3.9.0 and jetty-http 9.4.51.v20230217 inside database-plugins-*.jar. dependency:tree shows both come from hadoop-client 3.3.6 (compile scope) -> hadoop-yarn-client 3.3.6 (jline 3.9.0, websocket-client -> jetty-client -> jetty-http/-util/-io 9.4.51). The main code only uses org.apache.hadoop.{conf,io,util,mapreduce,mapred.lib.db} classes, which come from hadoop-common / hadoop-mapreduce-client-core that CDAP provides at runtime, so hadoop-client is now provided. This also stops packaging the rest of the Hadoop client tree (httpclient, commons-io 2.8.0, commons-compress 1.21, reload4j, kerby, curator, jersey-client, jackson jaxrs 2.12.7, ...) in the database-plugins jar. The root-managed avro-mapred 1.11.4 pulled avro-ipc and avro-ipc-jetty, which brought velocity-engine-core 2.3 (with commons-io 2.8.0) into the format-avro jar (and through it format-parquet / google-cloud). No plugin uses Avro IPC, so both are excluded. Versions: - hadoop-client 3.3.6: compile -> provided (database-plugins/pom.xml) - avro-ipc, avro-ipc-jetty 1.11.4: excluded from avro-mapred (root dependencyManagement) Bugs: Plugins side done for jline b/578444907, b/578489244, b/578492108, b/578644778, b/578663138 and jetty-http b/578515513. Risk: low. No main-code change; Hadoop classes are supplied by CDAP. Downstream projects that relied on database-plugins pulling hadoop-client transitively at compile scope must declare it themselves (none in this repo). Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl database-plugins,format-avro,format-parquet -am: BUILD SUCCESS; hydrator-common, format-common, format-avro, format-parquet and database-plugins unit tests pass (database-plugins: 35 run, 1 skipped: DBConnectorTest assumes live-DB system properties). dependency:tree -Dincludes=org.jline,org.eclipse.jetty,org.apache.avro:avro-ipc*,org.apache.velocity shows jline/jetty only at provided/test scope and no avro-ipc or velocity. unzip -l database-plugins jar: lib/ now only has hydrator-common, guava, opentracing-* and common-0.1.4 (no jline, jetty, hadoop-*); format-avro jar has no avro-ipc / velocity. DB source/sink against real MySQL / Postgres not run (needs a database). --- database-plugins/pom.xml | 2 ++ pom.xml | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/database-plugins/pom.xml b/database-plugins/pom.xml index 90524b353..d4b9fcf89 100644 --- a/database-plugins/pom.xml +++ b/database-plugins/pom.xml @@ -35,6 +35,8 @@ org.apache.hadoop hadoop-client + + provided io.cdap.cdap diff --git a/pom.xml b/pom.xml index 594766c6c..12a3bfc27 100644 --- a/pom.xml +++ b/pom.xml @@ -624,6 +624,15 @@ avro-mapred ${avro.version} + + + org.apache.avro + avro-ipc + + + org.apache.avro + avro-ipc-jetty + org.mortbay.jetty jetty From bd562808683b9076f41857e48be726ce12878b62 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:23:06 +0000 Subject: [PATCH 2/3] Bump log4j2, bcprov, commons-io, json-smart and iq80 snappy Image scanning of gcr.io/cdapio/cdap reports CVEs in jars packaged by hydrator-plugins: log4j-core 2.20.0 / log4j-api 2.21.1 (format-xls, core-plugins via poi), bcprov-jdk15to18 1.84 (core-plugins), commons-io 2.5 (format-delimited), json-smart 2.2.1 (transform-plugins via json-path 2.2.0) and iq80 snappy 0.2 (format-orc via orc-core 1.1.0). The root POM now imports log4j-bom and manages commons-io, json-smart and iq80 snappy, so modules no longer pin their own versions. json-path stays on 2.2.0 (2.9 would pull json-smart 2.5.x). Versions: - log4j2 (log4j-core, log4j-api) 2.20.0 / 2.21.1 -> 2.25.5 (root log4j2.version, log4j-bom import; format-xls log4j-core.version removed) - bcprov-jdk15to18 1.84 -> 1.85 (core-plugins) - commons-io 2.5 / 2.15.0 -> 2.16.1 (root commons-io.version + managed entry; format-delimited property and core-plugins / format-xls pins removed) - json-smart 2.2.1 -> 2.4.10 (root json-smart.version, managed) - iq80 snappy 0.2 -> 0.5 (root iq80-snappy.version, managed) Bugs: Plugins side done for log4j-core b/578431300, b/578491647, b/578514361, b/578514875. iq80 snappy is rows only (b/578567520 stays open through the Platform leveldb exception). Risk: low-medium. Same-major bumps. log4j 2.25.x and commons-io 2.16.1 still run on Java 8. commons-compress (1.26.0 in core-plugins / format-xls) is not touched in this PR. Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl core-plugins,format-xls,format-delimited,transform-plugins,format-orc -am: BUILD SUCCESS; core-plugins 328 tests (2 skipped), transform-plugins 100 (1 skipped), format-xls / format-delimited / format-orc and dependencies all pass. dependency:tree -Dincludes=org.apache.logging.log4j,org.bouncycastle,commons-io,net.minidev:json-smart,org.iq80.snappy shows only log4j 2.25.5, bcprov-jdk15to18 1.85, commons-io 2.16.1, json-smart 2.4.10 and snappy 0.5 at compile scope (smbj's bcprov-jdk15on 1.70 is runtime scope and is not embedded). unzip -l of the built jars: core-plugins lib/ has bcprov-jdk15to18-1.85, commons-io-2.16.1, log4j-api-2.25.5; format-xls has log4j-core/-api 2.25.5, commons-io-2.16.1; format-delimited commons-io-2.16.1; transform-plugins json-smart-2.4.10; format-orc snappy-0.5. Excel / ORC / SFTP pipelines in a sandbox not run. --- core-plugins/pom.xml | 3 +-- format-delimited/pom.xml | 2 -- format-xls/pom.xml | 3 --- pom.xml | 28 ++++++++++++++++++++++++++++ 4 files changed, 29 insertions(+), 7 deletions(-) diff --git a/core-plugins/pom.xml b/core-plugins/pom.xml index 74e7d5c49..8ffed57ca 100644 --- a/core-plugins/pom.xml +++ b/core-plugins/pom.xml @@ -136,7 +136,7 @@ org.bouncycastle bcprov-jdk15to18 - 1.84 + 1.85 @@ -196,7 +196,6 @@ commons-io commons-io - 2.15.0 org.apache.commons diff --git a/format-delimited/pom.xml b/format-delimited/pom.xml index c17c45571..b35a556b8 100644 --- a/format-delimited/pom.xml +++ b/format-delimited/pom.xml @@ -30,7 +30,6 @@ jar - 2.5 3.1 @@ -78,7 +77,6 @@ commons-io commons-io - ${commons-io.version} diff --git a/format-xls/pom.xml b/format-xls/pom.xml index c3ef74b6f..8d40cf018 100644 --- a/format-xls/pom.xml +++ b/format-xls/pom.xml @@ -27,7 +27,6 @@ jar 5.2.5 - 2.20.0 @@ -45,7 +44,6 @@ org.apache.logging.log4j log4j-core compile - ${log4j-core.version} io.cdap.cdap @@ -71,7 +69,6 @@ commons-io commons-io - 2.15.0 org.apache.commons diff --git a/pom.xml b/pom.xml index 12a3bfc27..f7dc69b85 100644 --- a/pom.xml +++ b/pom.xml @@ -88,6 +88,7 @@ 2.0.2.2 1.10 3.1 + 2.16.1 2.6 3.5 3.2.2 @@ -101,6 +102,7 @@ 13.0.1 3.3.6 2.2.4 + 0.5 1.4.1 4.13.1 2.24.0 @@ -111,6 +113,8 @@ 3.4.5 2.2.0 20160212 + 2.4.10 + 2.25.5 4.1.75.Final 1.3.0 3.3.2 @@ -133,6 +137,30 @@ + + org.apache.logging.log4j + log4j-bom + ${log4j2.version} + pom + import + + + commons-io + commons-io + ${commons-io.version} + + + + net.minidev + json-smart + ${json-smart.version} + + + + org.iq80.snappy + snappy + ${iq80-snappy.version} + org.slf4j slf4j-api From f3d6233a2a68f52978dbf8cceba26119f56b861d Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:25:05 +0000 Subject: [PATCH 3/3] Bump org.json to 20260719 Image scanning of gcr.io/cdapio/cdap reports org.json:json 20160212 in transform-plugins-*.jar (3 CVEs). transform-plugins is the only module that uses ${json.version}; only XMLToJSON uses org.json (JSONObject, XML.toJSONObject). Kept separate from the other minor bumps because newer org.json changes behavior: numbers are parsed as BigDecimal / BigInteger where needed, and XML.toJSONObject keeps strings with leading zeros and some other values that the old version coerced. Versions: - org.json:json 20160212 -> 20260719 (root json.version) Bugs: none closed alone in hydrator-plugins (the 3 org.json bugs close once WR-3b and GC-3b also land). Risk: medium. The XMLToJSON output can differ for numeric-looking values (for example leading zeros and very large / high-precision numbers). Testing: JDK 8, Checkstyle on (0 violations). mvn -DcloudBuild clean verify -pl transform-plugins -am: BUILD SUCCESS; transform-plugins 100 tests pass (1 skipped, ValueMapperTest, as on the HP-2a run), including XMLToJSONConverterTest (4), XMLParserTest (8), XMLMultiParserTest (2), JSONParserTest (8), JSONFormatterTest (2). unzip -l transform-plugins jar: lib/json-20260719.jar (no json-20160212). A sandbox pipeline with XMLToJSON was not run. --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index f7dc69b85..1dbb14f79 100644 --- a/pom.xml +++ b/pom.xml @@ -112,7 +112,7 @@ 4.0.3 3.4.5 2.2.0 - 20160212 + 20260719 2.4.10 2.25.5 4.1.75.Final