Repository navigation
Manage Jackson with jackson-bom 2.18.12 - #302
Open
deepika-lalwani123 wants to merge 3 commits into
Open
deepika-lalwani123 wants to merge 3 commits into
deepika-lalwani123 wants to merge 3 commits into
Conversation
The root POM declared ch.qos.logback:logback-classic 1.0.9 at compile
scope, so mysql-delta-plugins and sqlserver-delta-plugins packaged
logback-classic and logback-core 1.0.9 under lib/. Image scanning of
gcr.io/cdapio/cdap reports these jars under
/opt/cdap/master/artifacts/{mysql,sqlserver}-delta-plugins-*.jar/lib/.
No plugin main code uses logback; CDAP supplies the slf4j binding at
runtime. The only logback references are the logback-test.xml test
resources, which are ignored when no logback is on the test classpath.
delta-api 0.9.0 leaks junit 4.12 at compile scope (fixed at the source
in data-integrations/delta DL-1). Exclude it here so junit 4.12 is no
longer packaged (lib/junit-4.12.jar in mysql-delta-plugins), and add
junit 4.13.2 at test scope for the unit tests that relied on it.
Versions:
- logback-classic / logback-core 1.0.9 -> removed (root pom)
- junit 4.12 (compile, via delta-api) -> excluded; junit 4.13.2 test
scope (root pom)
Bugs: b/578434136, b/578581195. Plugins side done for the junit 4.12
row of b/578455300 (closes once DL-1 and DS-1 land).
Risk: low. Packaging-only change; no code change. Plugin logging still
goes through slf4j to the CDAP-provided binding.
Testing: JDK 8. mvn clean package -DtestSourceLocation=src/test/java
(develop never compiles src/test/java by default because
testSourceDirectory is ${testSourceLocation}): BUILD SUCCESS, Checkstyle
0 violations; delta-plugins-common RecordsTest, RuntimeArgumentsTest
(3 tests) and sqlserver SqlServerTableAssessorTest,
SqlServerRecordConsumerTest (7 tests) pass. mysql-delta-plugins has
only *IntegrationTest (Docker MySQL); integration and e2e tests not
run. mvn dependency:tree -Dincludes=junit,ch.qos.logback now shows only
junit:junit:4.13.2:test in every module (was logback-classic 1.0.9
compile and junit 4.12 compile). unzip -l of the built mysql/sqlserver
plugin jars shows no logback-*, junit-* or hamcrest-* under lib/
(develop: logback-classic/core 1.0.9 in both, junit-4.12 and
hamcrest-core-1.3 in mysql-delta-plugins).
mysql-delta-plugins and sqlserver-delta-plugins pin kafka-clients
3.9.1, which image scanning of gcr.io/cdapio/cdap reports in
/opt/cdap/master/artifacts/{mysql,sqlserver}-delta-plugins-*.jar/lib/
kafka-clients-3.9.1.jar. 3.9.2 is the fixed patch release and the last
Java 8 line.
kafka-clients 3.9.2 depends on at.yawk.lz4:lz4-java 1.10.1 (the
maintained lz4-java fork, same net.jpountz packages) instead of
org.lz4:lz4-java 1.8.0. Both are runtime scope and the bundle plugin
embeds only compile scope, so neither lz4 jar is packaged (before or
after) and no duplicate net.jpountz classes appear; no lz4 exclusion
is needed.
Versions:
- kafka-clients 3.9.1 -> 3.9.2 (mysql-delta-plugins,
sqlserver-delta-plugins)
Bugs: Plugins side done for the kafka-clients 3.9.1 rows (plan DBD-2;
the plan credits 0 bugs closed alone, 1 with other tracks).
Risk: low. Same-line patch release. debezium 1.9.7 keeps its Kafka
Connect 3.2.0 API jars; kafka-clients was already 3.9.1 next to them.
Testing: JDK 8. mvn -T 4 clean package -DtestSourceLocation=src/test/java:
BUILD SUCCESS, Checkstyle 0 violations, delta-plugins-common (3 tests)
and sqlserver-delta-plugins (7 tests) pass; Docker-based
*IntegrationTest and e2e not run. mvn dependency:tree
-Dincludes=org.apache.kafka:kafka-clients,org.lz4,at.yawk.lz4 on mysql
and sqlserver: kafka-clients 3.9.2 compile -> at.yawk.lz4:lz4-java
1.10.1 runtime; no 3.9.1. unzip -l: lib/kafka-clients-3.9.2.jar in both
plugin jars, no lz4 jar and no net/jpountz classes.
debezium 1.9.7.Final and Kafka Connect 3.2.0 bring a mix of old Jackson
jars (jackson-core 2.13.2, jackson-databind 2.13.2.2,
jackson-annotations / jackson-datatype-jdk8 2.12.6,
jackson-datatype-jsr310 2.13.2) that image scanning of
gcr.io/cdapio/cdap reports in
/opt/cdap/master/artifacts/{mysql,sqlserver}-delta-plugins-*.jar/lib/
(jackson-databind 13 rows and jackson-core 3 rows per plugin).
Import com.fasterxml.jackson:jackson-bom 2.18.12 in the root
dependencyManagement so every Jackson module resolves to 2.18.12.
debezium stays at 1.9.7.Final (2.x needs Java 11; see the debezium
exception in the plan).
Versions:
- jackson-core 2.13.2, jackson-databind 2.13.2.2, jackson-annotations
2.12.6, jackson-datatype-jdk8 2.12.6, jackson-datatype-jsr310 2.13.2
-> 2.18.12 (jackson-bom import, jackson.version)
Bugs: Plugins side done for the database-delta jackson rows (plan
DBD-3: 16 bugs touched, none closed by this PR alone).
Risk: medium. Jackson 2.15+ adds stream read limits and 2.18 changes
some introspection; debezium uses Jackson for its Document / history
JSON and Kafka Connect for JsonConverter. Both were exercised below.
Testing: JDK 8.
- mvn -T 4 clean package -DtestSourceLocation=src/test/java: BUILD
SUCCESS, Checkstyle 0 violations, delta-plugins-common (3) and
sqlserver-delta-plugins (7) unit tests pass.
- mvn clean verify -pl mysql-delta-plugins -am
-DtestSourceLocation=src/test/java (Docker mysql/mysql-server:8.0.13):
MySqlEventReaderIntegrationTest.test (snapshot + binlog CDC through
debezium) passes. stopReaderTest ("Reader never emitted any events")
and testBinaryHandlingModebyDebezium (IndexOutOfBounds) fail the same
way on develop (936c0a2), so they are pre-existing.
- Standalone smoke on the mysql plugin classpath: debezium
DocumentReader/DocumentWriter/HistoryRecord round-trip and Kafka
Connect JsonConverter struct round-trip work with Jackson 2.18.12.
- SQL Server integration tests not run (needs the 2017 MSSQL image).
- mvn dependency:tree -Dincludes=com.fasterxml.jackson.*: every Jackson
artifact in all modules is 2.18.12; unzip -l shows
lib/jackson-{annotations,core,databind,datatype-jdk8,datatype-jsr310}-2.18.12.jar
in both plugin jars and no older Jackson jar.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manage Jackson with jackson-bom 2.18.12
Repo: data-integrations/database-delta-plugins
Base: develop (stacked)
Commit: b4bd0a0
Part of the CVE fixes for the CDAP 6.11.2 patch release (container image scanning of gcr.io/cdapio/cdap).
debezium 1.9.7.Final and Kafka Connect 3.2.0 bring a mix of old Jackson
jars (jackson-core 2.13.2, jackson-databind 2.13.2.2,
jackson-annotations / jackson-datatype-jdk8 2.12.6,
jackson-datatype-jsr310 2.13.2) that image scanning of
gcr.io/cdapio/cdap reports in
/opt/cdap/master/artifacts/{mysql,sqlserver}-delta-plugins-*.jar/lib/
(jackson-databind 13 rows and jackson-core 3 rows per plugin).
Import com.fasterxml.jackson:jackson-bom 2.18.12 in the root
dependencyManagement so every Jackson module resolves to 2.18.12.
debezium stays at 1.9.7.Final (2.x needs Java 11).
Versions:
2.12.6, jackson-datatype-jdk8 2.12.6, jackson-datatype-jsr310 2.13.2
-> 2.18.12 (jackson-bom import, jackson.version)
Plugin-side updates are complete for the database-delta Jackson findings
(16 findings affected, none resolved by this PR alone).
Risk: medium. Jackson 2.15+ adds stream read limits and 2.18 changes
some introspection; debezium uses Jackson for its Document / history
JSON and Kafka Connect for JsonConverter. Both were exercised below.
Testing: JDK 8.
SUCCESS, Checkstyle 0 violations, delta-plugins-common (3) and
sqlserver-delta-plugins (7) unit tests pass.
-DtestSourceLocation=src/test/java (Docker mysql/mysql-server:8.0.13):
MySqlEventReaderIntegrationTest.test (snapshot + binlog CDC through
debezium) passes. stopReaderTest ("Reader never emitted any events")
and testBinaryHandlingModebyDebezium (IndexOutOfBounds) fail the same
way on develop (936c0a2), so they are pre-existing.
DocumentReader/DocumentWriter/HistoryRecord round-trip and Kafka
Connect JsonConverter struct round-trip work with Jackson 2.18.12.
artifact in all modules is 2.18.12; unzip -l shows
lib/jackson-{annotations,core,databind,datatype-jdk8,datatype-jsr310}-2.18.12.jar
in both plugin jars and no older Jackson jar.