From 89abaf8e37ff741cc34a6ec1e32cf37e0dba3adf Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:05:13 +0000 Subject: [PATCH 1/3] Move grpc-all to test scope to stop packaging unshaded Netty grpc-all 1.49.0 was a compile-scope dependency, so the datastream plugin jar packaged grpc-netty with unshaded Netty 4.1.77.Final (netty-buffer, -codec, -codec-http, -codec-http2, -common, -handler, -resolver, -transport), grpc-okhttp, grpc-rls, grpc-testing and, through grpc-testing, junit 4.12 and hamcrest-core 1.3. Image scanning of gcr.io/cdapio/cdap reports these under /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/ (40 rows). The exclusion meant to drop grpc-netty had groupId and artifactId swapped, so it never matched. Only DatastreamTableRegistryTest uses io.grpc directly (io.grpc.Status, which is in grpc-api), so grpc-all moves to test scope and the exclusion is fixed. The runtime transport stays grpc-netty-shaded. The plugin jar embeds only compile-scope dependencies, and gax-grpc declares grpc-auth, grpc-grpclb, grpc-services and grpc-xds at runtime scope; grpc-all used to lift them to compile scope. They are now declared directly at compile scope (versions from libraries-bom 26.1.1; grpc-xds, which that BOM does not manage, pinned to the same 1.48.0) so they stay in the plugin jar. As a side effect grpc-xds's own compile deps grpc-alts 1.48.0, opencensus-proto 0.2.0 and re2j 1.6 are now packaged too (they were runtime-only, so missing, before). delta-api's junit was already excluded in this repo. Versions: - grpc-all 1.49.0 compile -> test scope (exclusion fixed to io.grpc:grpc-netty) - io.netty 4.1.77.Final (8 jars), grpc-netty 1.48.0, grpc-okhttp 1.48.0, grpc-rls 1.49.0, grpc-testing 1.48.0, junit 4.12, hamcrest-core 1.3 -> no longer packaged - grpc-auth, grpc-grpclb, grpc-services, grpc-xds 1.48.0: now direct compile deps (unchanged versions) Bugs: plugin side for the unshaded Netty 4.1.77 rows (plan DS-1: 38 bugs touched, all also need other copies removed); junit b/578455300 closes once delta DL-1 and database-delta DBD-1 also land. Risk: low. No code change. The packaged gRPC set is the same except the removed test/alternate-transport jars and the three added xds deps. Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0 violations, 16 unit tests pass (DatastreamTableRegistryTest, DatastreamEventConsumerTest, DatastreamConfigTest, DatastreamAssessmentTest, UtilsTest; the credential-gated tests are skipped by their assumptions as on develop). mvn dependency:tree -Dincludes=io.netty,junit,org.hamcrest,io.grpc: no io.netty artifact at any scope; junit/hamcrest and grpc-okhttp/rls/testing only under grpc-all at test scope. unzip -l of the plugin jar vs develop: removed lib/netty-*-4.1.77.Final.jar (8), grpc-all, grpc-netty, grpc-okhttp, grpc-rls, grpc-testing, junit-4.12, hamcrest-core-1.3; added grpc-alts, opencensus-proto, re2j; lib/grpc-netty-shaded-1.48.0.jar kept. --- pom.xml | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/pom.xml b/pom.xml index f12eac6..acd9878 100644 --- a/pom.xml +++ b/pom.xml @@ -140,17 +140,41 @@ grpc-netty-shaded compile + + + io.grpc + grpc-auth + + + io.grpc + grpc-grpclb + + + io.grpc + grpc-services + + + io.grpc + grpc-xds + + 1.48.0 + io.grpc grpc-all 1.49.0 - io.grpc - grpc-netty + io.grpc + grpc-netty - compile + + test io.perfmark From 69054776167101165142e89ffe44054a83489f1b Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:06:54 +0000 Subject: [PATCH 2/3] Import jackson-bom 2.18.12 and a protobuf 3.25.5 floor libraries-bom 26.1.1 brings jackson-core 2.13.3 and protobuf-java / protobuf-java-util 3.21.4, which image scanning of gcr.io/cdapio/cdap reports in /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/ (jackson-core 3 rows, protobuf-java 4 rows). Import com.fasterxml.jackson:jackson-bom 2.18.12 and com.google.protobuf:protobuf-bom 3.25.5 before libraries-bom so they take precedence. Nothing that libraries-bom manages at a higher version is lowered: its protobuf (3.21.4) and jackson (2.13.3) are both older than the new floor. Upgrading libraries-bom itself is out of scope (WR-5-style change). Versions: - jackson-core 2.13.3 -> 2.18.12 (jackson-bom import, jackson.version) - protobuf-java, protobuf-java-util 3.21.4 -> 3.25.5 (protobuf-bom import, protobuf.version) Bugs: Plugins side done for the datastream jackson-core and protobuf rows (plan DS-2: 7 bugs touched, none closed by this PR alone). Risk: low-medium. protobuf 3.25 runtime runs code generated by 3.21 (the datastream v1 protos); jackson 2.15+ adds stream read limits, which only matter for very large JSON documents from GCS. Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0 violations, 16 unit tests pass. mvn dependency:tree -Dincludes=com.fasterxml.jackson.core,com.google.protobuf: only jackson-core 2.18.12 and protobuf-java / protobuf-java-util 3.25.5. unzip -l of the plugin jar vs DS-1: only those three jars changed version. --- pom.xml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/pom.xml b/pom.xml index acd9878..fd887c3 100644 --- a/pom.xml +++ b/pom.xml @@ -58,6 +58,8 @@ 1.7.25 1.8.2 26.1.1 + 2.18.12 + 3.25.5 2.3.3 0.8.8 @@ -80,6 +82,24 @@ + + + com.fasterxml.jackson + jackson-bom + ${jackson.version} + pom + import + + + com.google.protobuf + protobuf-bom + ${protobuf.version} + pom + import + com.google.cloud libraries-bom From 7c0ca2831817f3f93770b8ea4018dc55c56ef5c2 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:09:43 +0000 Subject: [PATCH 3/3] Bump avro from 1.8.2 to 1.11.4 avro 1.8.2 and its commons-compress 1.8.1 are reported by image scanning of gcr.io/cdapio/cdap in /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/ (avro 2 rows, commons-compress 6 rows; avro-1.8.2.jar also carries a shaded Guava 11 that is reported as Guava rows). avro 1.11.4 is the fixed release that still runs on Java 8 (1.12 needs Java 11). avro 1.9 dropped Jackson 1 (codehaus) and Joda from its API; the plugin only uses the generic reader API (DataFileReader, GenericDatumReader, GenericRecord, LogicalType) and compiles unchanged. Two packaging follow-ups: - avro 1.11.4 brings commons-compress 1.26.2, which pulls commons-lang3 3.14.0 (CVE-2025-48924); commons-lang3 is managed to 3.18.0. - snappy-java and xz are optional dependencies since avro 1.9, but 1.8.2 packaged them. They are declared directly (snappy-java at the cross-track 1.1.10.12, xz 1.9) so Avro files written with the snappy or xz codec stay readable. Versions: - avro 1.8.2 -> 1.11.4 (avro.version) - commons-compress 1.8.1 -> 1.26.2 (via avro), new commons-io 2.16.1 - commons-lang3 3.18.0 (managed; was not packaged) - snappy-java 1.1.1.3 -> 1.1.10.12, xz 1.5 -> 1.9 (direct) - jackson-core-asl / jackson-mapper-asl 1.9.13 and paranamer 2.7 -> no longer packaged; avro now uses jackson-databind 2.18.12 (DS-2 BOM) Bugs: b/578435773. Also removes the Guava 11 copy shaded in avro-1.8.2.jar (Guava bugs stay open as an exception, out of scope). Risk: low-medium (treated as MAJOR for testing). Crosses the avro 1.9 API break; no code change was needed. Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0 violations, 16 unit tests pass, including the 8 DatastreamEventConsumerTest cases that decode Avro records and logical types. mvn dependency:tree -Dincludes=org.apache.avro,org.apache.commons,commons-io,org.codehaus.jackson,org.xerial.snappy,org.tukaani: avro 1.11.4 -> commons-compress 1.26.2 -> commons-io 2.16.1, commons-lang3 3.18.0; snappy-java 1.1.10.12; xz 1.9; no codehaus jackson. unzip -l of the plugin jar vs DS-2 matches the list above. Not run: Datastream -> BigQuery replication against a real Datastream stream (needs GCP credentials; covered by wave smoke P4). --- pom.xml | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index fd887c3..93bea1f 100644 --- a/pom.xml +++ b/pom.xml @@ -56,7 +56,7 @@ 0.9.0 1.7.25 - 1.8.2 + 1.11.4 26.1.1 2.18.12 3.25.5 @@ -107,6 +107,12 @@ pom import + + + org.apache.commons + commons-lang3 + 3.18.0 + @@ -149,6 +155,20 @@ avro ${avro.version} + + + org.xerial.snappy + snappy-java + 1.1.10.12 + + + org.tukaani + xz + 1.9 + org.junit.jupiter junit-jupiter-engine