From aebbee6cf64eb209831f5afb2f636339876f1591 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:05:48 +0000 Subject: [PATCH 1/4] Exclude avro-ipc and avro-ipc-jetty from avro-mapred avro-mapred 1.11.4 pulls in avro-ipc and avro-ipc-jetty, which bring velocity-engine-core 2.3 (shaded commons-io 2.8.0, CVE-2024-47554) and org.tukaani:xz into lib/ of google-cloud-0.25.0-SNAPSHOT.jar, as seen in the image scan of gcr.io/cdapio/cdap (/opt/cdap/master/artifacts/google-cloud-*.jar/lib/velocity-engine-core-2.3.jar). The plugins only use org.apache.avro.mapred.AvroKey and org.apache.avro.mapreduce.* (BigQuery sink AvroOutputFormat / AvroRecordWriter, Dataplex tests). Avro RPC and tether jobs, the only users of avro-ipc, are not used. The Avro output codecs the plugins allow are snappy and deflate (FileSetUtil), so dropping xz is safe. Versions: - org.apache.avro:avro-ipc 1.11.4 -> removed (exclusion on avro-mapred) - org.apache.avro:avro-ipc-jetty 1.11.4 -> removed (exclusion on avro-mapred) - velocity-engine-core 2.3 and xz 1.9 no longer packaged Bugs: none closed alone. The other commons-io 2.8.0 copy (inside format-avro-2.13.0.jar) is removed by hydrator-plugins HP-1. Risk: very low. No plugin code path uses avro-ipc. Testing: JDK 8, Checkstyle on (0 violations). mvn clean verify: 647 tests, 0 failures, 5 errors, 8 skipped. The 5 errors are cloud ITs that pick up the Cloudtop's ambient GCP project and fail with PERMISSION_DENIED (GCSConnectorTest x2, SpannerConnectorTest x2, BigQueryConnectorTest); they fail the same way on develop. All other unit tests pass, including AvroRecordWriterTest and DataPlexOutputFormatProviderTest. mvn dependency:tree: avro-mapred:1.11.4 has no children; avro-ipc, avro-ipc-jetty, velocity-engine-core and xz are gone. unzip -l target/google-cloud-0.25.0-SNAPSHOT.jar: lib/ has no avro-ipc*, velocity-* or xz-* jar. --- pom.xml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/pom.xml b/pom.xml index 8451b89fc..5332c278a 100644 --- a/pom.xml +++ b/pom.xml @@ -613,6 +613,16 @@ avro-mapred ${avro.version} + + + org.apache.avro + avro-ipc + + + org.apache.avro + avro-ipc-jetty + com.fasterxml.jackson.core jackson-core From 85be4fa5ff8057f716496340c7b2258b07d4333b Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:11:56 +0000 Subject: [PATCH 2/4] Bump snappy-java, httpclient and httpcore patch versions Patch-level alignment with the cross-track targets used by the Platform CLs. No row of the current gcr.io/cdapio/cdap image scan points at these top-level jars in google-cloud, but they are below the versions used everywhere else and would re-open on the next scan: snappy-java 1.1.10.5 is below the 1.1.10.12 target, and httpcore 4.4.14 (transitive) did not match httpclient. 4.5.14 / 4.4.16 are what google-http-client-apache-v2 already uses, so nothing is downgraded. Versions: - org.xerial.snappy:snappy-java 1.1.10.5 -> 1.1.10.12 (direct dependency) - org.apache.httpcomponents:httpclient 4.5.13 -> 4.5.14 (httpclient.version, now also in dependencyManagement) - org.apache.httpcomponents:httpcore 4.4.14 -> 4.4.16 (new httpcore.version property, managed in dependencyManagement) Bugs: none (no scanned rows; prevents a re-open). Risk: very low. Same-line patch releases. Testing: JDK 8, Checkstyle on (0 violations). mvn clean verify: 647 tests, 0 failures, 5 errors, 8 skipped; the 5 errors are the same cloud ITs as on develop (GCSConnectorTest x2, SpannerConnectorTest x2, BigQueryConnectorTest: PERMISSION_DENIED on the Cloudtop's ambient GCP project, no test credentials). mvn dependency:tree: snappy-java 1.1.10.12, httpclient 4.5.14 and httpcore 4.4.16 at compile scope; no other version of them on the runtime classpath (httpcore-nio 4.4.5 stays test-only via cdap-unit-test). unzip -l target/google-cloud-0.25.0-SNAPSHOT.jar: lib/httpclient-4.5.14.jar, lib/httpcore-4.4.16.jar, lib/snappy-java-1.1.10.12.jar. --- pom.xml | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 5332c278a..c5809b631 100644 --- a/pom.xml +++ b/pom.xml @@ -79,7 +79,8 @@ 27.0.1-jre 3.3.6 2.6.6-hadoop3 - 4.5.13 + 4.5.14 + 4.4.16 2.13.4.2 4.13.1 2.0.2 @@ -102,6 +103,16 @@ google-api-services-storage v1-rev20211201-1.32.1 + + org.apache.httpcomponents + httpclient + ${httpclient.version} + + + org.apache.httpcomponents + httpcore + ${httpcore.version} + @@ -776,7 +787,7 @@ org.xerial.snappy snappy-java - 1.1.10.5 + 1.1.10.12 From aee054cdc05c281f447acb5098d64789d5a5e874 Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:18:16 +0000 Subject: [PATCH 3/4] Bump Jackson to 2.18.12, protobuf to 3.25.5, oauth-client to 1.33.3 The image scan of gcr.io/cdapio/cdap reports these top-level jars in lib/ of google-cloud-0.25.0-SNAPSHOT.jar: - jackson-core 2.12.3 (CVE-2025-49128, CVE-2025-52999, CVE-2026-68494, CVE-2026-89425), jackson-databind 2.13.4.2 (11 CVEs) and jackson-dataformat-cbor 2.9.9 (CVE-2020-28491) - protobuf-java 3.19.4 (CVE-2022-3171, CVE-2022-3509, CVE-2022-3510, CVE-2024-7254) - google-oauth-client 1.31.2 (CVE-2021-22573) Jackson is now aligned with an imported jackson-bom, so the cbor module that comes in through google-cloud-bigquery moves with core/databind. protobuf-bom aligns protobuf-java-util (3.17.3) with protobuf-java. No libraries-bom is imported here, so nothing brings a protobuf 4.x that could be downgraded. google-oauth-client 1.33.3 is built against google-http-client 1.41.x, so google-http-client-bom 1.41.8 is imported to keep the whole google-http-client family (core, jackson2, gson, apache-v2, appengine, protobuf) on one version instead of mixing 1.38.0 / 1.39.2 with 1.41. Versions: - jackson-core 2.12.3, jackson-databind / jackson-annotations 2.13.4.x, jackson-dataformat-cbor 2.9.9 -> 2.18.12 (jackson.core.version, now also used for the jackson-bom import) - protobuf-java 3.19.4 -> 3.25.5 (google.protobuf.java.version); protobuf-java-util 3.17.3 -> 3.25.5 (protobuf-bom import) - google-oauth-client 1.31.2 -> 1.33.3 (managed, new google.oauth.client.version) - google-http-client* 1.38.0 / 1.39.2 -> 1.41.8 (google-http-client-bom import, new google.http.client.version) The copies bundled inside bigtable-hbase-1.x-hadoop-1.17.1.jar (jackson-core 2.11.1, protobuf 3.12.2, oauth-client 1.31.0) are handled by GC-4; the Jackson copy inside format-avro-2.13.0.jar is handled in hydrator-plugins. Bugs: jackson-dataformat-cbor b/578686984 (closed by this change alone). Risk: medium. Minor-version jumps. Jackson 2.18 keeps the 2.x API but has stricter defaults (e.g. StreamReadConstraints limits on very large numbers / strings). protobuf 3.25 runtime is backward compatible with code generated by older 3.x protoc (grpc 1.39 / gax). Testing: JDK 8, Checkstyle on (0 violations). mvn clean verify: 647 tests, 0 failures, 5 errors, 8 skipped; the 5 errors are the same cloud ITs as on develop (GCSConnectorTest x2, SpannerConnectorTest x2, BigQueryConnectorTest: PERMISSION_DENIED on the Cloudtop's ambient GCP project, no test credentials). BigQuery, GCS, Spanner, Datastore, Pub/Sub, Dataplex unit tests pass. mvn dependency:tree: jackson-* 2.18.12, protobuf-java / protobuf-java-util 3.25.5, google-oauth-client 1.33.3, google-http-client* 1.41.8 at compile scope; 2.12.3 / 2.13.4.2 / 2.9.9 / 3.19.4 / 1.31.2 are gone. unzip -l target/google-cloud-0.25.0-SNAPSHOT.jar: lib/ has only the new versions (jackson-core-2.18.12, jackson-databind-2.18.12, jackson-dataformat-cbor-2.18.12, protobuf-java-3.25.5, google-oauth-client-1.33.3, google-http-client-1.41.8). Not run: e2e tests (need GCP credentials). --- pom.xml | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index c5809b631..81bc5ae85 100644 --- a/pom.xml +++ b/pom.xml @@ -64,6 +64,8 @@ 6.11.0 2.13.0 3.2.6 + 1.41.8 + 1.33.3 0.7.1 hadoop3-2.2.21 2.18.3 @@ -74,14 +76,14 @@ 1.24.7 2.3.0 1.105.1 - 3.19.4 + 3.25.5 1.3.0-rc3 27.0.1-jre 3.3.6 2.6.6-hadoop3 4.5.14 4.4.16 - 2.13.4.2 + 2.18.12 4.13.1 2.0.2 1.7.5 @@ -93,6 +95,35 @@ + + + com.fasterxml.jackson + jackson-bom + ${jackson.core.version} + pom + import + + + + com.google.protobuf + protobuf-bom + ${google.protobuf.java.version} + pom + import + + + + com.google.http-client + google-http-client-bom + ${google.http.client.version} + pom + import + + + com.google.oauth-client + google-oauth-client + ${google.oauth.client.version} + com.google.apis google-api-services-bigquery From d4b5af00dc8a839e12ef5cff11c31fb44f5d348a Mon Sep 17 00:00:00 2001 From: Deepika Lalwani Date: Tue, 17 Nov 2026 06:25:28 +0000 Subject: [PATCH 4/4] Bump org.json to 20260719 The image scan of gcr.io/cdapio/cdap reports org.json:json 20180813 (CVE-2022-45688, CVE-2023-5072, CVE-2026-59171) in lib/ of google-cloud-0.25.0-SNAPSHOT.jar. It comes in through tink 1.3.0-rc3 (and tink-gcpkms) and is used directly by GCSTinkTool and TinkDecryptor (JSONObject) to write and read the Tink encryption metadata of encrypted GCS files, and by Tink's JsonKeysetReader / JsonKeysetWriter. Kept separate from the other MINOR bumps because newer org.json releases change parsing behavior (stricter handling of malformed input, number conversion). A new unit test, TinkKeysetJsonTest, writes the metadata the same way GCSTinkTool does (JsonKeysetWriter -> JSONObject.toString(4)) and reads it back the same way TinkDecryptor does (new JSONObject -> getJSONObject("keyset") -> JsonKeysetReader), then decrypts a stream with the result, with a local AEAD key in place of Cloud KMS. It passes with both 20180813 and 20260719. Versions: - org.json:json 20180813 -> 20260719 (new org.json.version property, managed in dependencyManagement) Bugs: b/578504385, b/578575239, b/584531579 (closed once the hydrator-plugins HP-2b and wrangler WR-3b copies are also bumped). Risk: medium. org.json behavior changes; the Tink metadata round trip and Tink-encrypted read path are covered by the new test. Testing: JDK 8, Checkstyle on (0 violations). mvn clean verify: 648 tests, 0 failures, 5 errors, 8 skipped; the 5 errors are the same cloud ITs as on develop (GCSConnectorTest x2, SpannerConnectorTest x2, BigQueryConnectorTest: PERMISSION_DENIED on the Cloudtop's ambient GCP project, no test credentials). TinkKeysetJsonTest passes on this branch and with the old 20180813. mvn dependency:tree -Dincludes=org.json: tink:1.3.0-rc3 -> org.json:json:20260719 only. unzip -l target/google-cloud-0.25.0-SNAPSHOT.jar: lib/json-20260719.jar. Not run: GCS e2e with a real Cloud KMS key (needs GCP credentials). --- pom.xml | 7 ++ .../gcp/gcs/source/TinkKeysetJsonTest.java | 93 +++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 src/test/java/io/cdap/plugin/gcp/gcs/source/TinkKeysetJsonTest.java diff --git a/pom.xml b/pom.xml index 81bc5ae85..43d94b447 100644 --- a/pom.xml +++ b/pom.xml @@ -78,6 +78,7 @@ 1.105.1 3.25.5 1.3.0-rc3 + 20260719 27.0.1-jre 3.3.6 2.6.6-hadoop3 @@ -124,6 +125,12 @@ google-oauth-client ${google.oauth.client.version} + + + org.json + json + ${org.json.version} + com.google.apis google-api-services-bigquery diff --git a/src/test/java/io/cdap/plugin/gcp/gcs/source/TinkKeysetJsonTest.java b/src/test/java/io/cdap/plugin/gcp/gcs/source/TinkKeysetJsonTest.java new file mode 100644 index 000000000..8231b745e --- /dev/null +++ b/src/test/java/io/cdap/plugin/gcp/gcs/source/TinkKeysetJsonTest.java @@ -0,0 +1,93 @@ +/* + * Copyright © 2026 Cask Data, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ + +package io.cdap.plugin.gcp.gcs.source; + +import com.google.common.io.ByteStreams; +import com.google.crypto.tink.Aead; +import com.google.crypto.tink.JsonKeysetReader; +import com.google.crypto.tink.JsonKeysetWriter; +import com.google.crypto.tink.KeysetHandle; +import com.google.crypto.tink.StreamingAead; +import com.google.crypto.tink.aead.AeadConfig; +import com.google.crypto.tink.aead.AeadKeyTemplates; +import com.google.crypto.tink.streamingaead.StreamingAeadConfig; +import com.google.crypto.tink.streamingaead.StreamingAeadKeyTemplates; +import org.json.JSONObject; +import org.junit.Assert; +import org.junit.BeforeClass; +import org.junit.Test; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.Base64; + +import static java.nio.charset.StandardCharsets.UTF_8; + +/** + * Checks that the encryption metadata written by {@link GCSTinkTool} can be parsed with org.json and read back by + * Tink the same way {@link TinkDecryptor} does. A local AEAD key stands in for the Cloud KMS key. + */ +public class TinkKeysetJsonTest { + + private static final int ITERATIONS = 10; + + @BeforeClass + public static void setupClass() throws Exception { + AeadConfig.register(); + StreamingAeadConfig.register(); + } + + @Test + public void testMetadataRoundTrip() throws Exception { + Aead keysetAead = KeysetHandle.generateNew(AeadKeyTemplates.AES128_GCM).getPrimitive(Aead.class); + byte[] plaintext = "some plaintext to encrypt".getBytes(UTF_8); + byte[] associatedData = "gs://bucket/path/file.csv".getBytes(UTF_8); + + // Each iteration uses a new keyset, so different random key ids go through the JSON round trip. + for (int i = 0; i < ITERATIONS; i++) { + KeysetHandle handle = KeysetHandle.generateNew(StreamingAeadKeyTemplates.AES256_GCM_HKDF_4KB); + + // Encrypt and write the metadata the same way as GCSTinkTool. + ByteArrayOutputStream ciphertext = new ByteArrayOutputStream(); + try (OutputStream os = handle.getPrimitive(StreamingAead.class).newEncryptingStream(ciphertext, + associatedData)) { + os.write(plaintext); + } + ByteArrayOutputStream keyset = new ByteArrayOutputStream(); + handle.write(JsonKeysetWriter.withOutputStream(keyset), keysetAead); + JSONObject metadataObj = new JSONObject(); + metadataObj.put("kms", "gcp-kms://projects/p/locations/global/keyRings/r/cryptoKeys/k"); + metadataObj.put("aad", Base64.getEncoder().encodeToString(associatedData)); + metadataObj.put("keyset", new JSONObject(new String(keyset.toByteArray(), UTF_8))); + String metadataJson = metadataObj.toString(4); + + // Parse and read it back the same way as TinkDecryptor. + JSONObject metadata = new JSONObject(metadataJson); + KeysetHandle readHandle = KeysetHandle.read(JsonKeysetReader.withJsonObject(metadata.getJSONObject("keyset")), + keysetAead); + byte[] aad = Base64.getDecoder().decode(metadata.getString("aad")); + Assert.assertEquals(handle.getKeysetInfo().getPrimaryKeyId(), readHandle.getKeysetInfo().getPrimaryKeyId()); + + try (InputStream is = readHandle.getPrimitive(StreamingAead.class) + .newDecryptingStream(new ByteArrayInputStream(ciphertext.toByteArray()), aad)) { + Assert.assertArrayEquals(plaintext, ByteStreams.toByteArray(is)); + } + } + } +}