Repository navigation
Import jackson-bom 2.18.12 and a protobuf 3.25.5 floor - #156
Open
deepika-lalwani123 wants to merge 2 commits into
Open
deepika-lalwani123 wants to merge 2 commits into
deepika-lalwani123 wants to merge 2 commits into
Conversation
grpc-all 1.49.0 was a compile-scope dependency, so the datastream plugin jar packaged grpc-netty with unshaded Netty 4.1.77.Final (netty-buffer, -codec, -codec-http, -codec-http2, -common, -handler, -resolver, -transport), grpc-okhttp, grpc-rls, grpc-testing and, through grpc-testing, junit 4.12 and hamcrest-core 1.3. Image scanning of gcr.io/cdapio/cdap reports these under /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/ (40 rows). The exclusion meant to drop grpc-netty had groupId and artifactId swapped, so it never matched. Only DatastreamTableRegistryTest uses io.grpc directly (io.grpc.Status, which is in grpc-api), so grpc-all moves to test scope and the exclusion is fixed. The runtime transport stays grpc-netty-shaded. The plugin jar embeds only compile-scope dependencies, and gax-grpc declares grpc-auth, grpc-grpclb, grpc-services and grpc-xds at runtime scope; grpc-all used to lift them to compile scope. They are now declared directly at compile scope (versions from libraries-bom 26.1.1; grpc-xds, which that BOM does not manage, pinned to the same 1.48.0) so they stay in the plugin jar. As a side effect grpc-xds's own compile deps grpc-alts 1.48.0, opencensus-proto 0.2.0 and re2j 1.6 are now packaged too (they were runtime-only, so missing, before). delta-api's junit was already excluded in this repo. Versions: - grpc-all 1.49.0 compile -> test scope (exclusion fixed to io.grpc:grpc-netty) - io.netty 4.1.77.Final (8 jars), grpc-netty 1.48.0, grpc-okhttp 1.48.0, grpc-rls 1.49.0, grpc-testing 1.48.0, junit 4.12, hamcrest-core 1.3 -> no longer packaged - grpc-auth, grpc-grpclb, grpc-services, grpc-xds 1.48.0: now direct compile deps (unchanged versions) Bugs: plugin side for the unshaded Netty 4.1.77 rows (plan DS-1: 38 bugs touched, all also need other copies removed); junit b/578455300 closes once delta DL-1 and database-delta DBD-1 also land. Risk: low. No code change. The packaged gRPC set is the same except the removed test/alternate-transport jars and the three added xds deps. Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0 violations, 16 unit tests pass (DatastreamTableRegistryTest, DatastreamEventConsumerTest, DatastreamConfigTest, DatastreamAssessmentTest, UtilsTest; the credential-gated tests are skipped by their assumptions as on develop). mvn dependency:tree -Dincludes=io.netty,junit,org.hamcrest,io.grpc: no io.netty artifact at any scope; junit/hamcrest and grpc-okhttp/rls/testing only under grpc-all at test scope. unzip -l of the plugin jar vs develop: removed lib/netty-*-4.1.77.Final.jar (8), grpc-all, grpc-netty, grpc-okhttp, grpc-rls, grpc-testing, junit-4.12, hamcrest-core-1.3; added grpc-alts, opencensus-proto, re2j; lib/grpc-netty-shaded-1.48.0.jar kept.
libraries-bom 26.1.1 brings jackson-core 2.13.3 and protobuf-java / protobuf-java-util 3.21.4, which image scanning of gcr.io/cdapio/cdap reports in /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/ (jackson-core 3 rows, protobuf-java 4 rows). Import com.fasterxml.jackson:jackson-bom 2.18.12 and com.google.protobuf:protobuf-bom 3.25.5 before libraries-bom so they take precedence. Nothing that libraries-bom manages at a higher version is lowered: its protobuf (3.21.4) and jackson (2.13.3) are both older than the new floor. Upgrading libraries-bom itself is out of scope (WR-5-style change). Versions: - jackson-core 2.13.3 -> 2.18.12 (jackson-bom import, jackson.version) - protobuf-java, protobuf-java-util 3.21.4 -> 3.25.5 (protobuf-bom import, protobuf.version) Bugs: Plugins side done for the datastream jackson-core and protobuf rows (plan DS-2: 7 bugs touched, none closed by this PR alone). Risk: low-medium. protobuf 3.25 runtime runs code generated by 3.21 (the datastream v1 protos); jackson 2.15+ adds stream read limits, which only matter for very large JSON documents from GCS. Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0 violations, 16 unit tests pass. mvn dependency:tree -Dincludes=com.fasterxml.jackson.core,com.google.protobuf: only jackson-core 2.18.12 and protobuf-java / protobuf-java-util 3.25.5. unzip -l of the plugin jar vs DS-1: only those three jars changed version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Import jackson-bom 2.18.12 and a protobuf 3.25.5 floor
Repo: data-integrations/datastream-delta-plugins
Branch: vuln/ds-2-jackson-protobuf-floor → base develop (stacked)
Commit: 6905477
Part of the CVE fixes for the CDAP 6.11.2 patch release (container image scanning of gcr.io/cdapio/cdap).
libraries-bom 26.1.1 brings jackson-core 2.13.3 and protobuf-java /
protobuf-java-util 3.21.4, which image scanning of gcr.io/cdapio/cdap
reports in /opt/cdap/master/artifacts/datastream-delta-plugins-*.jar/lib/
(jackson-core 3 rows, protobuf-java 4 rows).
Import com.fasterxml.jackson:jackson-bom 2.18.12 and
com.google.protobuf:protobuf-bom 3.25.5 before libraries-bom so they
take precedence. Nothing that libraries-bom manages at a higher
version is lowered: its protobuf (3.21.4) and jackson (2.13.3) are
both older than the new floor. Upgrading libraries-bom itself is out of
scope (similar to data-integrations/wrangler#1074).
Versions:
import, protobuf.version)
Plugin updates are complete for the datastream jackson-core and protobuf
findings (not fully resolved by this PR alone).
Risk: low-medium. protobuf 3.25 runtime runs code generated by 3.21
(the datastream v1 protos); jackson 2.15+ adds stream read limits,
which only matter for very large JSON documents from GCS.
Testing: JDK 8. mvn clean package: BUILD SUCCESS, Checkstyle 0
violations, 16 unit tests pass. mvn dependency:tree
-Dincludes=com.fasterxml.jackson.core,com.google.protobuf: only
jackson-core 2.18.12 and protobuf-java / protobuf-java-util 3.25.5.
unzip -l of the plugin jar vs #155: only those three jars changed
version.