Repository navigation
test(j2k-native): check whole codestreams for marker values in tile data - #320
Merged
jcwal1516 merged 1 commit intoNov 3, 2026
Merged
Conversation
T.800 B.10.7: "The sequence of bytes actually included for any given code-block must not end in a 0xFF." Code-block data and packet headers follow one another directly, so a trailing 0xFF pairs with the next byte, and a pair in 0xFF90..=0xFFFF is a marker value (D.4, note). frames-sg#167 drops the classic MQ encoder's terminal 0xFF, and its tests check each code-block's bytes. These check what the rule protects, on whole encoded codestreams: - tile_data_holds_no_marker_values: no byte-aligned 0xFF90..=0xFFFF pair and no final 0xFF in any tile-part's data, for 16x9, 64x64 and 256x256 images, reversible and irreversible, classic and HTJ2K. The reversible codestreams decode back to their samples. - packet_bodies_never_end_in_0xff: with SOP and EPH, no packet body ends in 0xFF, so the next packet's SOP stays a marker. Both fail with the terminal-0xFF drop in ArithmeticEncoder::finish_checked removed (a byte-aligned 0xFFCF in the 16x9 reversible classic tile data, and a packet body ending in 0xFF) and pass on main. AI-Tooling: Claude Opus 5.5 <noreply@anthropic.com>
Member
|
Thanks for catching this! I ran both new tests on my current checkout and they passed. LGTM, I'll merge once CI is green. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#167 drops the classic MQ encoder's terminal 0xFF, as T.800 B.10.7 requires ("The sequence of bytes actually included for any given code-block must not end in a 0xFF"). Its tests check each code-block's bytes:
finish_omits_pending_terminal_ff,short_sequences_never_end_in_ff_and_round_tripandclassic_code_block_contributions_never_end_in_ff.Nothing checks what the rule protects on a whole codestream. Code-block data and packet headers follow one another directly, so a trailing 0xFF pairs with the next byte, and a byte-aligned pair in 0xFF90–0xFFFF is a marker value (D.4, note). A decoder resynchronising on SOP/EPH, or looking for the EOC after a tile-part with Psot 0, can stop at one.
Change
One integration test file,
crates/j2k-native/tests/codeblock_termination.rs, and no library changes. It encodes through the publicencodeand finds each tile-part's data from its SOT's Psot and its SOD.tile_data_holds_no_marker_values: no byte-aligned 0xFF90–0xFFFF pair and no final 0xFF in any tile-part's data. It covers 16×9, 64×64 and 256×256 grey images, reversible and irreversible, classic and HTJ2K. The reversible codestreams must also decode back to their samples.packet_bodies_never_end_in_0xff: with SOP and EPH on, no packet body ends in 0xFF, which would otherwise pair with the next packet's SOP.The samples are noise over a gradient, so code-blocks have enough coding passes that most MQ terminations would end in 0xFF without the fix.
Results
Both tests pass on
mainat 968dbb4. With theself.data.pop()inArithmeticEncoder::finish_checkedremoved, both fail:No changelog entry, since the change is test-only (like #177 and #318).
Run locally (aarch64 macOS):
cargo fmt --all --checkxtask's pedantic and nursery lints and baseline allowscargo xtask repo-lintcargo test --release -p j2k-native -p j2k-codec-math -p j2k: 1,661 passed, 0 failedI wrote these tests with the same encoder fix on my fork; #167 landed first, so only the tests are left to offer.
AI-Tooling: Claude Opus 5.5 noreply@anthropic.com