Repository navigation
fix(j2k): align packet validation across decode routes and backends - #317
Merged
Merged
Conversation
…every route
The packet walk of decodes on the component planes failed with
PacketParseFailure("packet header outlives the progression iterator") when
a tile-part held bytes after the tile's last packet, and with "packet
length markers contain unused entries" when its PLT listed more packets
than the tile has. Region decodes, and the full-image decodes that stream
their rows through them, stop after the last packet they need and decoded
the same codestreams. The full-image route follows the pool's size, so a
half-resolution decode of a 512 x 512 RGB image with one byte after its
last packet failed on one thread and decoded on four.
The walk now stops at the end of the tile's progression and leaves the
rest of the tile-part unread, and PLT lengths left after the last packet
are accepted. PLT lengths left while the progression continues in a later
tile-part remain an error. The output equals that of the codestream
without the extra data, as OpenJPEG 2.5 decodes these inputs. On a corpus
of 4160 malformed and valid codestreams, full-resolution decodes now give
the same result on the planes and streamed routes (305 inputs differed);
valid codestreams parse the same packets as before.
…us PLT lengths The end-of-tile-part PLT check takes the next packet from the progression to tell lengths left after the tile's last packet from lengths of packets still to come. A walk that goes on after an error, as a non-strict one does, then lost that packet and parsed the next tile-part from the one after it. Only walks that stop at an error look ahead now; strict walks, the ones every public decode runs, are unchanged. Tests: a surplus PLT length in a tile-part followed by more of the tile's packets fails on the planes, streamed and default routes at full and half resolution on one and four threads; a tile's last packet with its non-empty bit cleared still fails on every route when PLT lengths are signalled; a non-strict walk records the same segments with and without a surplus length in its first tile-part (it lost the next tile-part's packets before).
…cket Direct plans, which the GPU decodes and the staged batch execute, read packets with the same walk as the decode routes. A test now builds the direct RGB plan of classic and HTJ2K codestreams with a byte after the tile's last packet and with a PLT length past it: each plan builds, the one with the extra byte is the clean codestream's plan, and both, run on the CPU, write the clean codestream's decoded image. Before the walk stopped at the end of the progression, building either plan failed.
…ty tests Edits of raw codestreams around their packets move to the shared test support crate, so the facade, native and GPU tests build the same inputs: `edit_tile_parts` (Psot kept consistent), a byte after each tile-part's last packet, an extra PLT length, the packet byte ranges a tile-part's PLT lengths give, and packets overwritten in place. The facade route tests use them; their inputs and assertions are unchanged.
Decodes below full resolution on the component planes parsed every packet of the tile. The streamed route parses with a region decode's precinct selection: the packets of the resolutions it keeps, skipping the others by their PLT lengths where every packet has one, and none after the last it needs. A codestream damaged only in packets of the discarded resolutions therefore decoded or failed depending on the route, which follows the pool size: on a corpus of 4160 malformed and valid codestreams, 216 half-resolution decodes failed on the planes route and decoded streamed. For each of them that can be checked (155, single tile), the streamed output equals the half-resolution decode of the codestream cut after the last packet that decode needs. Planes decodes below full resolution now parse with the region route's selection for the whole reduced image, made on a scratch storage and keeping every precinct's record, so both routes read the same packets; the build is unchanged. The selection charges the decode budget for what it allocates beside the tile storage's retained capacity, not for the precinct records the build later makes in that capacity, so it rejects no tile that the build accepts. Damage in a packet the reduced output needs, or in one that must be parsed to locate a later needed packet, remains an error, and full-resolution decodes are unchanged.
…des read Direct plans below full resolution, which the GPU decodes and the staged batch execute, parsed every packet of the tile, so a codestream damaged only in packets of the discarded resolutions failed to plan where the CPU decodes at that resolution succeed. They now parse with the same selection as the planes and streamed decodes, so every backend accepts and rejects the same codestreams; plans of valid codestreams are unchanged, as they only ever held the kept resolutions' code-blocks.
Half-resolution decodes of codestreams damaged in chosen packets, on the planes, streamed and default routes on pools of one and four threads: finest-resolution packets damaged in RPCL (with PLT, without, over tiles and over several tile-parts per tile), RLCP HTJ2K, and LRCP, CPRL and PCRL with PLT lengths to skip them decode as the undamaged codestream does; the same damage in packets that must be parsed to reach a later needed one (LRCP without PLT, with a progression order change, or with one tile-part lacking PLT lengths), in the coarsest packets, and every full-resolution decode of these codestreams fail. A PLT length that ends a skipped packet inside the next needed one or past the tile-part fails, and a tile-part cut anywhere in its last packet decodes at half resolution. In the native crate, cutting or damaging an RPCL tile's last packet gives each of the packet parse errors the planes route reported at half resolution before (PLT disagreement, short body, coding-pass count, inclusion tree, segment length, EPH), and the planes, region and direct plan decodes at half resolution all write the undamaged image; damage in a needed packet fails all three; and planes decodes below full resolution parse exactly the packet headers the region decode parses. The reduced and region decode docs state what such decodes validate.
The forced-route tests kept only whether each decode failed, so a route that failed for another reason than the others went unnoticed. Each decode now gives the class of the native error behind its failure (or the facade's own error, which a failing case must not have), and every route, pool size and planes budget of one codestream and request must fail with one class. All cases do.
…uld skip by A walk that skips unneeded packets by their PLT lengths, as region decodes and decodes below full resolution do, took any length as the packet's. Every packet with an in-band header holds at least the first byte of its header, and two more for its EPH marker where the coding style requires one, so a shorter length made the walk read the next packet from the bytes of the one it skipped. In a flat three-component CPRL image whose finer packets are empty, a zero length for the second component's finest packet and a length of one for the third component's coarsest made a half-resolution decode read the empty finest packet as the third component's coarsest, find every packet it needs and stop: it reported success with RGB (88, 88, 158) where the image is (200, 50, 120), on the planes and the streamed route alike. With EPH markers and three resolutions, lengths of one and two for the second component's finer packets did the same at quarter resolution. Full decodes reject these lengths, as the packets they parse disagree with them. Skipping now fails with a packet error on a length below one byte, or three where EPH markers are required. Lengths a walk never skips by (after its last needed packet, or with packed packet headers, whose PLT lengths cover the body alone) are not examined. Facade tests check every route, pool size and the direct plan at every resolution of both images.
…-image selection The whole-image selection that planes decodes and direct plans below full resolution parse with charged only what it allocates, without the region selection's up-front check of the build's structure, so as not to count the retained capacity the build reuses twice. A tile the build must reject was then walked first: a 105-byte header declaring one 16,383 x 16,383 gray tile with one-sample LL precincts and two-sample ones above (2^28 sub-band precincts) made a half-resolution direct plan hold 134 MB and walk 2^27 precincts before the build failed it. The selection now first checks that the decode's live bytes, the selection and the structure the build stores for every precinct (one record and two tag-tree roots each, and the skeleton) fit the cap, without the retained capacity, which the build reuses for them, and without the region build's ROI windows. The region selection's check is unchanged. The tile above now fails with a resource limit before any precinct is walked: the direct plan holds under 4 MiB, the planes decode nothing beyond the coefficient plane it allocates before building the tile.
…ngths too short to skip by
…ion decodes do A precinct can hold no code-block, as at a one-sample tile corner whose detail sub-bands are empty. Region decodes, and full-image decodes that stream their rows, leave its packets unread when they come after the last packet the output needs, or skip them by their PLT lengths; full-image decodes on the component planes and direct plans read every packet. A 3 x 3 image in 2 x 2 tiles with EPH markers whose last packet, that of the corner tile's empty precinct, has a damaged EPH marker therefore failed on the planes route and decoded streamed, at full resolution; on Metal it failed on explicit requests and tile batches and decoded on the batch decoder, which plans the image as a region. Planes decodes and direct plans of a tile with such a precinct now parse with the whole-image region selection, as below full resolution; other full-resolution decodes keep parsing every packet without a selection. Only the last precinct of a resolution above 0 can lack code-blocks, as precincts there are at least two samples wide and high and start at even coordinates, so the check reads one precinct per resolution however many the tile declares; a unit test compares it with a check of every precinct over 3600 tile geometries. The facade and Metal tests check every route, pool size and Metal entry point at full and half resolution.
…every route With packet headers in PPM or PPT, PLT lengths cover packet bodies only, so reduced decodes parse every header up to their last needed packet. In a flat image whose later headers are one empty byte each, damage to a finest packet's header before a needed packet fails every route and pool size at half resolution, and damage to the last header decodes there as the undamaged codestream; both fail at full resolution. Tile-parts of several tiles taking turns, one RPCL packet each, with twelve 64 x 64 tiles, which stream as one region, and eight 512 x 512 ones, which stream tile by tile at once at half resolution on four threads (a route test checks that choice): the interleaved codestream decodes as the tile-major one; damage to every tile's finest packet decodes at half resolution as the undamaged codestream, and damage to every coarsest packet fails; both fail at full resolution. The test support gains the interleaving and a lookup of the packed headers.
…lans A multiple component transform needs three components. Decodes of a one-component codestream whose COD asks for one fail with a colour error in strict mode, but the grayscale direct plan and the referenced grayscale plans (classic and HTJ2K), which GPU decodes and decode batches execute, never looked at the flag and decoded such a codestream: a 39 x 7 one-bit gray image in five tiles with the flag set failed every decode of `J2kDecoder` and decoded in the prepared batch on the CPU and on CUDA. The plans now fail with the same error where a tile asks for the transform and the header is strict, once every tile they cover is planned, as decodes fail after parsing every tile's packets: a failure a decode meets first, such as a precinct grid past the decode cap, keeps its class. A native test checks the single-tile direct plan and the referenced plan over four tiles against the decode's error class, and that precedence on a 16,383 x 16,383 tile at half resolution.
Metal full and reduced-resolution decodes build their direct plans, or fall back to decoding the component planes, with the packet walk the CPU decodes use. Device tests now hold every Metal entry point that consumes it (explicit requests with and without a session, the tile batch and the batch decoder) to the CPU decode on six fixtures: gray and RGB, classic and HTJ2K, one tile and four, at full and half resolution. - Bytes or PLT lengths after a tile's last packet decode to the CPU output of the codestream without them. - A last packet shorter than its PLT length, a PLT length left while the tile continues in a later tile-part, and a damaged first packet fail on Metal with the native error class the CPU decode reports. - At half resolution, damage confined to finest-level packets after the last needed one (resolution-major, with or without PLT lengths) or skipped by PLT length (layer-major) decodes to the clean image; damage in a needed packet, or in one parsed to reach a needed packet, and any of it at full resolution, fails on both with one error class.
…ry point Extends the Metal packet-walk parity tests with the facade's newer cases, each checked on the CPU decode and on every Metal entry point (request, session request, tile batch with a distinct companion, and the batch decoder for 8-bit output), 8- and 16-bit, classic and HT where the fixture allows: - a zero PLT length for an empty finest packet before a needed one, in a flat CPRL RGB image, fails at full and half resolution with the CPU's error class, and the codestream with its true lengths decodes as on the CPU; - with EPH markers, PLT lengths of one and two for empty finer packets before a needed one fail at full, half and quarter resolution likewise; - with PPM or PPT packet headers, damage to a finest header before a needed packet fails at half resolution, damage to the last header decodes there as the undamaged codestream, and both fail at full resolution; - tile-parts taking turns, one RPCL packet each, in the tiled fixtures and in a 2048 x 1024 gray image of eight 512 x 512 tiles, decode as the tile-major codestream; damage to every tile's finest packets decodes at half resolution as the undamaged codestream, damage to the coarsest fails, and both fail at full resolution. Without the minimum PLT lengths in the packet walk, the CPU decode and every Metal entry point decode the zero-length codestream at half resolution and the short-length one at quarter resolution. Without the series' native changes, the CPU decode fails the reduced cases that every Metal entry point already decoded to the undamaged image.
A code-block's raw (bypass) coding passes may read past the coded bytes of their terminated segment when the segment holds fewer bits than the passes consume, as a truncated or lossily coded codestream can leave it. T.800 D.4.1 extends such a cleanly exhausted segment with 0xFF bytes, so every raw bit past its end reads as one, in strict mode as in lenient mode; the CPU decoder does this, but the Metal classic kernel's raw bit reader failed the read in strict mode, so a decode the CPU completed by padding failed on Metal with a code-block decode error. The kernel's raw bit reader now returns a one bit at exhaustion regardless of strict mode, as the CPU decoder does. Strict mode still rejects a missing or set stuff bit after a 0xFF byte, which the stuffing check makes separately, so malformed byte stuffing stays an error on both. A regression test shortens a raw segment of a bypass code-block so its passes exhaust it and asserts the scalar decode and the Metal kernel pad to the same samples; it fails before this change with a kernel decode error.
Codestreams with data after a tile's last packet, PLT lengths past it, or damaged packets decode on CUDA as they decode on the CPU, through the single-image decode, the batch of images and the prepared batch, at full and half resolution, gray and RGB, classic and HTJ2K: the samples of the clean codestream where the CPU decode leaves the extra data or the damaged packets unread, the CPU decode's native error where it reads them. The half-resolution cases cover damage confined to finest-level packets with and without PLT lengths, in RPCL and LRCP order, and damage in a packet the reduced decode needs. A tiled image whose last packet, that of a precinct without code-blocks, has a damaged EPH marker decodes as the clean image on the prepared batch; the single-image CUDA decodes, whose direct plans take one tile, reject it as unsupported before reading any packet, as they reject the clean image.
…rs and interleaved tile-parts Extend the CUDA packet-walk parity tests with the cases the CPU facade tests cover, on the single-image decode, the batch of images and the prepared batch: - a zero PLT length for a packet a half-resolution decode skips, before a packet it needs (flat RGB in CPRL, flat gray in LRCP with two layers), and PLT lengths of one and two where EPH markers make every packet at least three bytes, which fail at full, half and quarter resolution; - damaged PPM and PPT packet headers: before a needed packet the half-resolution decode fails, in the last header it decodes as the undamaged codestream, and both fail at full resolution; - tile-parts of several tiles interleaved, six 32 x 32 tiles of every kind and eight 512 x 512 tiles of a 2048 x 1024 gray image. Direct plans take one tile, so the single-image CUDA decodes reject these as unsupported; the prepared batch decodes them on the device as the CPU does. Every CUDA outcome must equal the CPU decode's: the same samples, or the same native decode error. A CUDA prepared batch whose group fails on the device now reports that error as its outcome. With the packet walk from before the length check, the zero-length and EPH cases decode at half and quarter resolution on the CPU and every CUDA route, and the tests fail.
…native error A strict decode rejects a code-block whose packet headers claim more missing bit-planes than its sub-band codes, or more coding passes than its remaining bit-planes hold. The CPU decode reports these as InvalidBitplaneCount and TooManyCodingPasses, decode errors of the codestream. The CUDA classic Tier-1 plan rejected the same blocks as an unsupported CUDA request, so the single-image decode, the batch of images and the prepared batch failed with another error class than the CPU decode of the same codestream. The plan now checks these two conditions for strict blocks as the CPU decode does and fails with a native decode error of the class the CPU decode reports. Where a codestream has more than one fault, the first error each reports can still differ. Lenient blocks, and the plan's own geometry and segment checks, keep the unsupported-request rejection. A packet-walk test lowers the QCD guard bits of lossless classic codestreams by one, which leaves every coded block one bit-plane's passes too many: before the change every CUDA route failed with the unsupported request where the CPU decode failed with TooManyCodingPasses. Unit tests cover the strict and lenient checks.
The CPU batch decoder hands the rows of a streamed image to a sink that converts every sample to the batch's u8 output through a function pointer, in f64. The decode calls that sink on the thread that reads its rows, between the batches it reconstructs, and meanwhile the pool's band workers decode only as far ahead as their band rings hold. The conversion was therefore serial time on the decode's path. Since the finest band rings of classic tiles read per component hold two bands in place of three, the band workers run out of room sooner during it: a 5661 x 8287 RGB image decoded at half resolution as a one-image batch on 18 threads took 79 ms in place of 71 ms, while the same decode into a caller's RGB8 buffer, which packs its rows on the pool, got no slower. The band plan is the same on both routes (three readers, the same rings and parts); only the hand-over differs. The sink now converts a row at a time with the same rounding and saturation in f32. f32::round is exact and, above zero, equals the f64 floor of the sample plus one half, and values past the precision's maximum saturate, so the bytes are the same. The compiler can vectorize the contiguous rows of planar output, and interleaved rows convert in a tight loop. A unit test compares the row conversion with the per-sample one around every rounding and saturation boundary, at NaN and the infinities, for precisions 1 to 16, planar and interleaved. Measured on an 18-core ARM64 machine, 9 interleaved rounds against two builds of the base, default threads, output identical, peak heap unchanged: - the 5661 x 8287 RGB image as a one-image batch at half resolution: 79.2 -> 64.9 ms (71.2 ms before the shallower rings), at full resolution 196 -> 127 ms; four and eight copies with automatic workers 592 -> 447 ms and 1065 -> 876 ms; - the same image decoded alone at half resolution, into a buffer or as a region: unchanged (1.008, 0.997); - eight distinct 2048 x 2048 RGB (interleaved) or 2896 x 2896 gray images on 1, 2, 4, 8 and automatic workers: 1.5-3.5% faster in every case. With the conversion fast, the shallower rings no longer cost the batch time: at half resolution the batch takes 64.5 ms with them and without them, and keeps their 3 MiB lower peak.
Borrow split prepared groups and share checked native-output sizing, plane validation and stores. Remove redundant owned-batch admission claims while preserving the staged-window budget and direct-ROI admission. Reuse the streaming parity driver with typed errors and padding checks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some codestreams decoded or failed according to the selected CPU route, thread count or GPU backend. Align packet selection and validation across those paths, and fix tile-level marker overrides and CUDA region error propagation. This follows #316 and contains the packet-validation and batch-conversion changes described below.
Nine interleaved rounds on an 18-core ARM64 machine measured the 5661×8287 RGB one-image batch at half resolution at 79.2 → 64.9 ms and at full resolution at 196 → 127 ms. Output and peak heap were unchanged. Boundary tests cover the row converter’s precision, rounding, saturation, NaN and infinity behavior.
This PR adds no public API beyond its parent. It leaves the #133 ring-depth policy, whole-window error rerun, band-release rule and serial executor unchanged.
Changed files with prior jcoyne contributions (12)
Validation
Packet and tile-marker regressions, CUDA classic-plan tests, batch unit/integration tests, streaming parity tests and xtask self-tests pass. Formatting, strict and upstream clippy, stable API and repository lint pass. Existing route tests compare output and typed failures across serial/pool execution and reduced/full requests.
The full macOS suite passes on the combined stack tip: 5,656 passed, 0 failed, 52 ignored. This PR changes GPU execution and planning. Its own exact-head CUDA and Metal quick validation passed on this PR’s exact head.