Repository navigation
sbe: keep fixed-length byte arrays through XML import and export - #184
Merged
Merged
Conversation
xmlToProto ignored the length of a non-char array, so a uint8 array of 16 imported as one uint32 narrowed to uint8 and every later field in the block moved. A uint8 or int8 array now imports as bytes with the same (sbe.length), and an array of a wider primitive is refused, naming the type, field and message. protoToXml wrote a bytes field as a char array, which imports back as a string. It now writes a uint8 array, declared once per length as bytesN. The wire bytes do not change. A bytes field of length 1 exports as length="1", which FIX SBE reads as a single uint8, so it comes back as a narrowed uint32; pinned by a test. Follows protowire-go#251; the schema and cases are its tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #179.
The SBE XML converter lost fixed-length arrays that are not text. This ports protowire-go#251, following the rule in draft -01 § Fixed-Length Arrays.
Import (
Convert.xmlToProto)uint8orint8array of length N > 1 becomesbyteswith(sbe.length) = N. Before, it became one narroweduint32, so a 16-byte field was read as 1 byte and every later field in the block moved.IllegalArgumentException, naming the type, the field and the message. For example:sbe: message "Record": field "uuid": type "Uuid" is an array of 2 uint64, which no proto field represents. Go uses the same text.Export (
Convert.protoToXml)bytesfield with(sbe.length) = Nis written asprimitiveType="uint8" length="N", declared once per length asbytesN. Before, it was achararray, which imports back as astring. Astringstays achararray.bytesfield of length 1 comes back asuint32 [(sbe.encoding) = "uint8"]: the same byte in the block, but notbytes. A test pins this.The wire bytes do not change. Only the converter's output does.
Tests (
FixedArrayTest, using Go's schema and cases)arrays/arr.protoand compiled. A test checks it equals what the importer writes now.Evidence
./gradlew :sbe:test: 41 tests, 0 failures.FixedArrayTestran 9.Convert.javareverted to main, 7 of the 9 fail. The two that still pass test the codec against the checked-in proto, not the converter../gradlew buildwith the plugin fetched at the pin (-Dorg.gradle.project.pxfJavaMeta.repo=/nonexistent, as CI does): BUILD SUCCESSFUL.