Skip to content

Move Google Cloud clients to libraries-bom with grpc-bom 1.85.1 - #1074

Open
deepika-lalwani123 wants to merge 6 commits into
developfrom
vuln/wr-5-libraries-bom-grpc
Open

deepika-lalwani123 wants to merge 6 commits into
developfrom
vuln/wr-5-libraries-bom-grpc

Conversation

@deepika-lalwani123

@deepika-lalwani123 deepika-lalwani123 commented Nov 17, 2026 •

Copy link
Copy Markdown

Move Google Cloud clients to libraries-bom with grpc-bom 1.85.1

Stacked PR: this branch is built on top of #1069, #1070, #1071, #1072, #1073. Against develop the diff includes those commits; please review only the top commit 818bbf6f2f. Merge after #1073.

Repo: data-integrations/wrangler
Branch: vuln/wr-5-libraries-bom-grpc → base develop (stacked)
Commit: 818bbf6
Part of the CVE fixes for the CDAP 6.11.2 patch release (container image scanning of gcr.io/cdapio/cdap).

Image scanning of gcr.io/cdapio/cdap reports 42 rows in
grpc-netty-shaded-1.31.1.jar (shaded Netty 4.1.48: codec-http, -http2,
handler, ...) inside the wrangler-service (DataPrep) jar. It came from
google-cloud-spanner 1.61.0 and the other pinned 1.x Google Cloud
clients (bigquery 1.110.1, storage 1.106.0, core 1.93.4). They are now
managed by com.google.cloud:libraries-bom, with io.grpc:grpc-bom
1.85.1 imported first so grpc (and its shaded Netty 4.2.17) is newer
than what the BOM ships (grpc 1.83.0).

Both BOMs are imported in wrangler-service only, the one module that
uses Google Cloud clients. In the root pom libraries-bom also manages
Guava, which moved the Guava 13 that the CDAP unit-test framework
needs to 33.x and broke 12 wrangler-storage tests
(IncompatibleClassChangeError).

Versions (wrangler-service):

  • google-cloud-spanner 1.61.0 -> 6.127.0, google-cloud-bigquery
    1.110.1 -> 2.76.0, google-cloud-storage 1.106.0 -> 2.78.0,
    google-cloud-core 1.93.4 -> 2.80.0, google-oauth-client 1.34.1 ->
    1.39.0 (libraries-bom 26.93.0; google.cloud.*.version,
    google.storage.version, google.oauth.client.version removed)
  • grpc 1.31.1 -> 1.85.1 (new grpc.version, grpc-bom), incl.
    grpc-netty-shaded (shaded Netty 4.1.48 -> 4.2.17.Final)
  • protobuf-java(-util) 3.25.5 -> 4.33.6 in wrangler-service only (new
    protobuf4.version; the clients are generated for protobuf 4; the
    other modules stay on 3.25.5)
  • guava 31.0.1-jre -> 33.6.0-jre in wrangler-service only (from the
    BOM; gax 2.90 / grpc 1.85 need Guava 33)
  • gax 1.54 -> 2.90, google-auth-library 0.20 -> 1.57,
    google-http-client 1.34 -> 2.2.0, gson 2.10.1 -> 2.14.0 (BOM)
  • New in the jar: arrow 17 (+ netty-buffer/netty-common
    4.1.140.Final, via google-cloud-bigquery), bigquerystorage,
    monitoring, opentelemetry, grpc-xds and others; lib/ grows from 172
    to 215 jars

No source change was needed: wrangler-service main and test code
compile against Spanner 6.x, BigQuery 2.x and Storage 2.x as is.

Plugin changes are done for the Netty findings.

Risk: high. Major-version jumps of the GCS, BigQuery and Spanner
clients (new transports, retries, defaults) in the DataPrep
connections; Guava 33 and protobuf 4 in the app classloader. All jars
in lib/ are Java 8 bytecode (checked: no class file above major 52).
Needs the GCS / BigQuery / Spanner connection checks (browse,
sampling, proxy) in a 6.11 sandbox (P3) before release.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped, same as develop), Transform 2/0.
dependency:tree -Dincludes=io.grpc on wrangler-service: every io.grpc
artifact is 1.85.1 (grpc-netty-shaded 1.85.1); no 1.31.1 left.
unzip -l lib/: grpc-netty-shaded-1.85.1.jar (its
io.netty.versions.properties says 4.2.17.Final), no grpc 1.31.1 or
Netty 4.1.48 jar. Offline smoke run on the wrangler-service runtime
classpath: StorageOptions/BigQueryOptions/SpannerOptions with
NoCredentials build their services, Spanner admin clients,
Statement binding, Type.Code, Date and Timestamp work (no linkage
errors). Live GCS / BigQuery / Spanner calls not run (no credentials
here).

Image scanning of gcr.io/cdapio/cdap reports logback-classic and
logback-core 1.2.11 inside the wrangler-service (DataPrep system app)
jar. Wrangler does not declare logback and no wrangler code imports
ch.qos.logback; it came in only as a compile dependency of
io.cdap.cdap:cdap-etl-core (confirmed with dependency:tree, the only
path). CDAP provides the logging backend at runtime, so the jar does
not need its own copy.

Versions:
- logback-classic / logback-core 1.2.11 -> not packaged (exclusion of
  ch.qos.logback:* on cdap-etl-core in wrangler-service)
- logback-classic 1.2.13 added at test scope only (logback.version in
  wrangler-service), declared before hadoop-mapreduce-client-core so it
  stays the SLF4J binding for unit tests; without it slf4j-reload4j
  binds and FileTypeDetectorTest fails with ExceptionInInitializerError

Bugs: Plugins side done for b/578430422, b/578585447, b/578633383,
b/578635398, b/578643563, b/578657694, b/578688736 (the logback CVEs
fixed only in 1.3.x/1.5.x stay blocked by the Platform logback
exception).

Risk: low. Only the packaged jar list changes; logging in the app goes
through slf4j to the CDAP-provided backend as for other plugins.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean verify -pl wrangler-service: Tests run: 17, Failures: 0,
Errors: 0, Skipped: 6 (same as develop). dependency:tree
-Dincludes=ch.qos.logback on wrangler-service shows only
logback-classic/core 1.2.13 at test scope. unzip -l of
wrangler-service-4.12.0-SNAPSHOT.jar: lib/ no longer has
logback-classic-1.2.11.jar or logback-core-1.2.11.jar; no other lib/
change. DataPrep sandbox logging check (P3) not run here.
Patch-level bumps for CVEs that image scanning of gcr.io/cdapio/cdap
reports in the wrangler-service and wrangler-transform jars
(httpclient 4.5.3, avro 1.11.3). httpcore was not pinned: it resolved
to 4.4.13 (via google-cloud-spanner) in wrangler-service and 4.4.6 in
wrangler-core / wrangler-transform; it is now managed at 4.4.16 to
match httpclient 4.5.14. netty is used only at test scope in wrangler
(netty-all, netty-http); the bump keeps the netty-bom import aligned
with the cross-repo target and changes no packaged jar.

Versions:
- httpclient 4.5.3 -> 4.5.14 (httpclient.version, now also in root
  dependencyManagement)
- httpcore 4.4.13 / 4.4.6 -> 4.4.16 (new httpcore.version, root
  dependencyManagement)
- avro 1.11.3 -> 1.11.4 (avro.version); avro 1.11.4 also moves its
  jackson-core/databind/annotations from 2.14.2 to 2.14.3
- netty-bom 4.1.118.Final -> 4.1.140.Final (netty.version)

Bugs: none close alone. Plugins (wrangler) side done for httpclient
b/578423875 (also needs google-cloud GC-4 and Platform); the avro bugs
also need KP-3 / DS-3.

Risk: low. Same-line patch releases. avro 1.11.4 rejects untrusted
java-class schema properties (CVE-2024-47561 fix); wrangler does not
use avro reflect/specific classes from schema props.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped, same as develop), Transform 2/0.
dependency:tree -Dincludes=org.apache.httpcomponents,org.apache.avro
on wrangler-core/-service/-transform: only httpclient 4.5.14, httpcore
4.4.16, avro 1.11.4 at compile scope. unzip -l: wrangler-service and
wrangler-transform lib/ now have httpclient-4.5.14, httpcore-4.4.16,
avro-1.11.4 (old jars gone).
Minor-version bumps for CVEs that image scanning of gcr.io/cdapio/cdap
reports in the wrangler-service and wrangler-transform jars. Where the
library is transitive it is pinned in the root dependencyManagement.
Real sources (checked with dependency:tree; the plan guessed some):
- commons-beanutils 1.9.3 comes from org.everit.json.schema 1.11.1
  (wrangler-core), not from commons-validator
- commons-compress 1.25.0 comes from poi-ooxml 5.2.5
- json-smart 2.2.1 comes from json-path 2.2.0
- log4j-api 2.17.2 comes from poi 5.2.5 (log4j-bom import)

Versions:
- commons-compress 1.25.0 -> 1.26.2 (new commons-compress.version, DM)
- commons-lang3 3.5 -> 3.18.0 (commons-lang3.version; needed by
  compress 1.26 and itself a CVE fix)
- commons-beanutils 1.9.3 -> 1.11.0 (new commons-beanutils.version, DM)
- bcprov-jdk18on 1.78.1 -> 1.85 (bouncycastle.version)
- log4j 2.17.2 -> 2.25.5 (log4j.version, log4j-bom import)
- json-smart 2.2.1 -> 2.4.10 (new json-smart.version, DM); this also
  moves accessors-smart 1.1 -> 2.4.9
- protobuf-java 3.11.3 -> 3.25.5 (protobuf.version); protobuf-java-util
  is now managed at the same version (was 3.11.4 via the Google Cloud
  clients in wrangler-service)

Bugs: closes b/578429610, b/578444267, b/578489829, b/578574083,
b/583757186 (bcprov) and b/578581625 (json-smart). Plugins side done
for b/578439719, b/578443850, b/578572706, b/578584043, b/578584206,
b/578645797.

Risk: medium. commons-lang3 jumps 3.5 -> 3.18 (source compatible;
deprecations only). protobuf 3.25 runtime runs the older 3.x gencode of
the Google Cloud 1.x clients. accessors-smart 2.4.9 declares asm 9.3
but org.ow2.asm:asm 5.0.4 (via kryo/reflectasm, nearer) still wins;
checked at runtime that json-smart bean mapping
(JSONValue.parse(json, Bean.class)), JSONValue.toJSONString and
JsonPath.read work on the wrangler-core runtime classpath with
asm 5.0.4.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped, same as develop), Transform 2/0.
dependency:tree on wrangler-core/-service/-transform (compile scope):
commons-compress 1.26.2, commons-lang3 3.18.0, commons-beanutils
1.11.0, bcprov-jdk18on 1.85, log4j-api 2.25.5, json-smart 2.4.10,
protobuf-java(-util) 3.25.5; no old version left. unzip -l lib/ of
wrangler-service and wrangler-transform shows the same new jars and
none of the old ones.
Image scanning of gcr.io/cdapio/cdap reports org.json:json 20180813 in
the wrangler-service and wrangler-transform jars. It is a direct
dependency of wrangler-core (directives use JSONObject, JSONTokener,
JSONException and XML: parse-xml-to-json, JsParser, json conformer,
regex filter, fill-null-or-empty). Kept separate from the other minor
bumps because newer org.json changes parsing behavior.

Versions:
- org.json:json 20180813 -> 20260719 (new org.json.version; wrangler-core
  and root dependencyManagement, which also aligns the copy that
  org.everit.json.schema asks for)

Behavior change: XML.toJSONObject now returns a BigDecimal for numbers
with an exponent, so parse-xml-to-json renders 303246306303E8 as
3.03246306303E+19 instead of 3.03246306303E19 (same value). The
XmlToJsonTest expectation is updated for that; with keepStrings=true
the output is unchanged.

Bugs: wrangler copy done for b/578504385, b/578575239, b/584531579
(they close when hydrator-plugins HP-2b and google-cloud GC-3b land
too).

Risk: medium. Newer org.json parses numbers more exactly (BigDecimal /
BigInteger) and is stricter about malformed input; all wrangler-core
directive tests pass.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped), Transform 2/0. JSON/XML directive tests: XmlToJsonTest 1,
JsonConformerTest 5, RowRegexFilterTest 4, FillNullOrEmptyTest 5,
WriteAsJsonMapTest 1, WriteAsJsonObjectTest 1, JsonFunctionsTest 2,
SchemaConverterTest 5, JsParserTest, JsPathTest: all pass.
XmlToJsonTest failed before the expectation update (E19 vs E+19 only).
dependency:tree -Dincludes=org.json: only json 20260719 at compile
scope in wrangler-core/-service/-transform; unzip -l lib/ of both
plugin jars shows json-20260719.jar and no json-20180813.jar.
Image scanning of gcr.io/cdapio/cdap reports jackson-core 2.7.9 and
jackson-core/databind 2.14.2 in the wrangler-service and
wrangler-transform jars. Wrangler declares no Jackson dependency and
has no com.fasterxml imports; every copy is transitive (checked with
dependency:tree):
- jackson-core 2.7.9: com.microsoft.azure:azure-data-lake-store-sdk
  2.3.3 (wrangler-service), not from aws-java-sdk-s3 as assumed; it was
  packaged next to databind 2.14.x, i.e. a mismatched pair
- jackson-core/databind/annotations 2.14.x: org.apache.avro:avro
  (wrangler-core)
- jackson-dataformat-cbor 2.12.6: aws-java-sdk-core (wrangler-service)
Importing the BOM in the root dependencyManagement aligns all of them.
The jackson-databind exclusion on aws-java-sdk-s3 in wrangler-service
stays.

Versions:
- jackson-core 2.7.9 / 2.14.3 -> 2.18.12
- jackson-databind, jackson-annotations 2.14.3 -> 2.18.12
- jackson-dataformat-cbor 2.12.6 -> 2.18.12
  (new jackson.version, com.fasterxml.jackson:jackson-bom import)

Bugs: none close here alone; this finishes the wrangler copies of the
Jackson rows (they also need hydrator-plugins HP-4, google-cloud GC-4
and Platform).

Risk: medium. Jackson 2.18 still targets Java 8 and is API compatible
for the users here (avro, google-http-client-jackson2, AWS SDK, ADLS
SDK); 2.18 adds default stream read constraints (max string/number
length) that only affect very large documents.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped), Transform 2/0.
dependency:tree -Dincludes=com.fasterxml.jackson.*: only 2.18.12 at
compile scope in wrangler-core/-service/-transform. unzip -l lib/:
wrangler-service has jackson-annotations/core/databind/dataformat-cbor
2.18.12 (jackson-core-2.7.9, databind 2.14.x and cbor 2.12.6 gone);
wrangler-transform has annotations/core/databind 2.18.12 only.
S3/ADLS/GCS connection checks in a sandbox (P3) not run here.
Image scanning of gcr.io/cdapio/cdap reports 42 rows in
grpc-netty-shaded-1.31.1.jar (shaded Netty 4.1.48: codec-http, -http2,
handler, ...) inside the wrangler-service (DataPrep) jar. It came from
google-cloud-spanner 1.61.0 and the other pinned 1.x Google Cloud
clients (bigquery 1.110.1, storage 1.106.0, core 1.93.4). They are now
managed by com.google.cloud:libraries-bom, with io.grpc:grpc-bom
1.85.1 imported first so grpc (and its shaded Netty 4.2.17) is newer
than what the BOM ships (grpc 1.83.0).

Both BOMs are imported in wrangler-service only, the one module that
uses Google Cloud clients. In the root pom libraries-bom also manages
Guava, which moved the Guava 13 that the CDAP unit-test framework
needs to 33.x and broke 12 wrangler-storage tests
(IncompatibleClassChangeError).

Versions (wrangler-service):
- google-cloud-spanner 1.61.0 -> 6.127.0, google-cloud-bigquery
  1.110.1 -> 2.76.0, google-cloud-storage 1.106.0 -> 2.78.0,
  google-cloud-core 1.93.4 -> 2.80.0, google-oauth-client 1.34.1 ->
  1.39.0 (libraries-bom 26.93.0; google.cloud.*.version,
  google.storage.version, google.oauth.client.version removed)
- grpc 1.31.1 -> 1.85.1 (new grpc.version, grpc-bom), incl.
  grpc-netty-shaded (shaded Netty 4.1.48 -> 4.2.17.Final)
- protobuf-java(-util) 3.25.5 -> 4.33.6 in wrangler-service only (new
  protobuf4.version; the clients are generated for protobuf 4; the
  other modules stay on 3.25.5)
- guava 31.0.1-jre -> 33.6.0-jre in wrangler-service only (from the
  BOM; gax 2.90 / grpc 1.85 need Guava 33)
- gax 1.54 -> 2.90, google-auth-library 0.20 -> 1.57,
  google-http-client 1.34 -> 2.2.0, gson 2.10.1 -> 2.14.0 (BOM)
- New in the jar: arrow 17 (+ netty-buffer/netty-common
  4.1.140.Final, via google-cloud-bigquery), bigquerystorage,
  monitoring, opentelemetry, grpc-xds and others; lib/ grows from 172
  to 215 jars

No source change was needed: wrangler-service main and test code
compile against Spanner 6.x, BigQuery 2.x and Storage 2.x as is.

Bugs: Plugins side done for the Netty bugs b/578422530, b/578425552,
b/578426660, b/578427732, b/578435805, b/578435812, b/578442798,
b/578452844, b/578498484, b/578501458, b/578511634, b/578512623,
b/578520387, b/578523920, b/578524138, b/578525371, b/578559603,
b/578566449, b/578568144, b/578571769, b/578631548, b/578636943,
b/578637410, b/578646421, b/578646479, b/578655606, b/578663185.

Risk: high. Major-version jumps of the GCS, BigQuery and Spanner
clients (new transports, retries, defaults) in the DataPrep
connections; Guava 33 and protobuf 4 in the app classloader. All jars
in lib/ are Java 8 bytecode (checked: no class file above major 52).
Needs the GCS / BigQuery / Spanner connection checks (browse,
sampling, proxy) in a 6.11 sandbox (P3) before release.

Testing: JDK 8, Checkstyle on (0 violations).
mvn clean install (all modules): BUILD SUCCESS. Tests: API 31/0 fail,
Proto 3/0, Core 456/0 (2 skipped), Storage 37/0, Service 17/0
(6 skipped, same as develop), Transform 2/0.
dependency:tree -Dincludes=io.grpc on wrangler-service: every io.grpc
artifact is 1.85.1 (grpc-netty-shaded 1.85.1); no 1.31.1 left.
unzip -l lib/: grpc-netty-shaded-1.85.1.jar (its
io.netty.versions.properties says 4.2.17.Final), no grpc 1.31.1 or
Netty 4.1.48 jar. Offline smoke run on the wrangler-service runtime
classpath: StorageOptions/BigQueryOptions/SpannerOptions with
NoCredentials build their services, Spanner admin clients,
Statement binding, Type.Code, Date and Timestamp work (no linkage
errors). Live GCS / BigQuery / Spanner calls not run (no credentials
here).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates and aligns multiple dependency versions across the project's POM files (including pom.xml, wrangler-core/pom.xml, and wrangler-service/pom.xml) to address CVEs and ensure compatibility. Key changes include upgrading Bouncy Castle, Netty, Log4j, Protobuf, and Jackson, as well as introducing BOMs for Google Cloud libraries and gRPC. A unit test in XmlToJsonTest.java was also updated to match the exponent formatting of the newer org.json library. All review comments were filtered out as they pertained to version numbers, so there is no additional feedback to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Triggers unit test build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants