JBIG2 + JPEG 2000: decode every file from issue #1

docs/known-gaps.md

@@ -53,7 +53,7 @@ HEIF 解码也已调优（M3），基准项目是 `Sdcb.SimdCodecs.Heif.Benchmar
   0..255（线性光数据如增益图 HDR 再加 `srgbEncode: true`），但不做色调映射。
 - **多帧读取器的帧数可能少于文件里的帧数。** `ImageInfo.FrameCount` 报告文件里有多少帧或页，但：
   - APNG：解码还没实现，读取器只给出 PNG 的默认图（1 帧）；
-  - JBIG2：独立 `.jb2` 文件只解码第 1 页，读取器只有 1 帧。
+  - JBIG2：门面的读取器只给出独立 `.jb2` 文件的第 1 页（1 帧）；其他页用 `Jbig2Decoder.DecodeFile(file, pageNumber)`。
 - **接口里没有“时间戳 → 帧号”的辅助方法。** WebP 包的 `WebpAnimationDecoder.FrameIndexAt` 有，接口层没有；
   需要时可以按各帧的 `Duration` 自己累加。
 - **`IImageFrameReader` 和 `ImageDecoder` 会话都不是线程安全的**（动画需要在调用之间保留画布状态），每个线程各开一个。

docs/known-gaps/jbig2.md

@@ -2,10 +2,19 @@
 
 条目约定（“不支持”“门面不提供”“未调优”）和跨格式的门面限制见 [总览](../known-gaps.md)。
 
-- **独立 `.jb2` 文件只解码第 1 页。** PDF 内嵌的 JBIG2 流没有文件头可供识别，需要直接调用
-  `Jbig2Decoder.Decode(embedded, globals, width, height)`，门面不处理。
+- **门面只解码独立 `.jb2` 文件的第 1 页。** 其他页用 `Jbig2Decoder.DecodeFile(file, pageNumber)`，页数由
+  `Jbig2Decoder.TryReadFileInfo(file, out w, out h, out pages)` 给出。PDF 内嵌的 JBIG2 流没有文件头可供识别，
+  需要直接调用 `Jbig2Decoder.Decode(embedded, globals, width, height)`，门面不处理。
 - **不支持**：
   - EXTTEMPLATE 通用区域；
-  - 使用 HENABLESKIP 的半色调区域；
-  - MMR 编码的半色调区域和图案字典。
+  - 引用或保留算术上下文的符号字典（符号字典标志的 bit 8、9）；
+  - MMR 的 2D 扩展码（含 T.6 非压缩模式）。
+- **Huffman 编码只有部分覆盖。** 标准表、自定义表段、未压缩和 MMR 的高度类集合位图、Huffman 文本区域，都有
+  `Jbig2ExtendedOracleTests` 用手工组装的码流对照 jbig2dec。Huffman 符号字典里的精细化 / 聚合符号（SDHUFF 与
+  SDREFAGG 同时为 1）、Huffman 文本区域里的逐实例精细化，没有黑盒编码器能造流，只有 issue #1 语料里的
+  `042_14.jb2` 用到，而 jbig2dec 和本解码器都判它为坏流，所以这两条路径没有经过对照验证。标准表 B.7 的码长
+  之和是 15/16（不满），也只在单元测试里检查过。
+- issue #1 附件里的 114 个 `.jb2` 文件（来源和许可不明，不进仓库）由 `Issue1Jbig2CorpusTests` 在本地回归：
+  设置 `SIMDCODECS_ISSUE1_DIR` 指向解压目录即可。112 个和 jbig2dec 逐页逐像素一致；`042_13`、`042_14` 两个
+  jbig2dec 和本解码器都在同一处判为坏流，本解码器抛 `InvalidDataException`。
 - 未调优。

docs/known-gaps/jpeg2000.md

@@ -2,14 +2,22 @@
 
 条目约定（“不支持”“门面不提供”“未调优”）和跨格式的门面限制见 [总览](../known-gaps.md)。
 
-- **只支持 Part 1（T.800），定位是 PDF 里的 `/JPXDecode`。**
+- **只支持 Part 1（T.800），定位是 PDF 里的 `/JPXDecode`。** 多 tile、tile-part、自定义 precinct、
+  五种进度顺序、SOP / EPH 标记、JP2 调色板（`pclr` + `cmap`）都已支持。
 - **不支持**：
   - 进度顺序变更（POC）；
   - 打包的包头（PPM、PPT）；
   - 感兴趣区域（RGN / MAXSHIFT）；
-  - 在 tile-part 头里覆盖主头参数的标记。
-- **分量位深最多 16 位，且所有分量位深必须相同。**
+  - 在 tile-part 头里覆盖主头参数的标记（COD / COC / QCD / QCC）；
+  - code-block 样式里的选择性旁路（0x01）、每个 pass 单独终止（0x04）、垂直因果上下文（0x08）。
+    上下文重置（0x02）、可预测终止（0x10）、分段符号（0x20）已支持；
+  - JP2 的通道定义盒（`cdef`）；
+  - 分量子采样、带符号分量。
+- **分量位深最多 16 位，且所有分量位深必须相同**；经调色板映射后的各通道位深也必须相同。
 - **门面只呈现 1 或 3 个分量、且不是 sYCC 的图像**；带 alpha、CMYK、sYCC 的图门面返回 `false`，
   `Jpeg2000Decoder.Decode` 仍会返回全部分量。
-- **没有分辨率缩减（reduce）解码。**
-- **已有基准测试，性能待调优**：基准项目 `Sdcb.SimdCodecs.Jpeg2000.Benchmarks` 已建立并在 `results/` 记录初始基线；目前纯托管单核解码耗时约为 libopenjp2 纯 C 核心的 1.9×~2.2×；EBCOT 条带位掩码快速跳过、Code-block 级多线程并发与系数重构缓冲区复用尚未实现。
+- **门面不提供分辨率缩减（reduce）**；`Jpeg2000Decoder.Decode(data, reduce)` 可以直接调用。
+- **真实文件回归**：issue #1 附带的 16 个 JPEG 2000 文件（来源和许可不明，不进仓库）全部与
+  `opj_decompress` 一致（5/3 逐样本相同，9/7 最多差 1）。改动解码器后按 `Issue1Jpeg2000CorpusTests`
+  的说明下载并跑一遍。
+- **已有基准测试，性能待调优**：基准项目 `Sdcb.SimdCodecs.Jpeg2000.Benchmarks` 已建立并在 `results/` 记录初始基线；目前纯托管单核解码耗时约为 libopenjp2 纯 C 核心的 1.9×~2.2×；EBCOT 条带位掩码快速跳过、Code-block 级多线程并发与系数重构缓冲区复用尚未实现。tile 之间是串行解码的。

src/Sdcb.SimdCodecs.Jbig2/GenericRegionDecoder.cs

@@ -97,6 +97,11 @@ internal static class GenericRegionDecoder
     /// <param name="budget">
     /// The decode's remaining pixel allowance, charged before anything is allocated.
     /// </param>
+    /// <param name="skip">
+    /// USESKIP / SKIP of T.88 §6.2.5.7: where this bitmap holds a 1 the pixel is not
+    /// coded at all and is set to 0. Null when USESKIP is 0. Must be
+    /// <paramref name="width"/> x <paramref name="height"/>.
+    /// </param>
     public static Jbig2Bitmap Decode(
         ref MqDecoder mq,
         scoped Span<byte> contexts,
@@ -105,8 +110,11 @@ public static Jbig2Bitmap Decode(
         int template,
         bool typicalPrediction,
         scoped ReadOnlySpan<sbyte> at,
-        Jbig2PixelBudget budget)
+        Jbig2PixelBudget budget,
+        Jbig2Bitmap? skip = null)
     {
+        if (skip is not null && (skip.Width != width || skip.Height != height))
+            throw new ArgumentException("The SKIP bitmap must match the region size.", nameof(skip));
         var expectedAt = template == 0 ? 8 : 2;
         if (at.Length < expectedAt)
             throw new ArgumentException($"GBTEMPLATE {template} needs {expectedAt / 2} AT pixel(s).", nameof(at));
@@ -138,8 +146,22 @@ public static Jbig2Bitmap Decode(
             }
 
             var row = y * width;
-            for (var x = 0; x < width; x++)
-                bitmap.Data[row + x] = (byte)mq.Decode(contexts, Context(bitmap, x, y, template, at));
+            if (skip is null)
+            {
+                for (var x = 0; x < width; x++)
+                    bitmap.Data[row + x] = (byte)mq.Decode(contexts, Context(bitmap, x, y, template, at));
+            }
+            else
+            {
+                // §6.2.5.7 step 3 c): a skipped pixel costs no decision and reads as 0.
+                // The bitmap is freshly zeroed, so leaving it alone is setting it to 0.
+                var skipped = skip.Data;
+                for (var x = 0; x < width; x++)
+                {
+                    if (skipped[row + x] != 0) continue;
+                    bitmap.Data[row + x] = (byte)mq.Decode(contexts, Context(bitmap, x, y, template, at));
+                }
+            }
         }
 
         return bitmap;

src/Sdcb.SimdCodecs.Jbig2/HalftoneDecoder.cs

@@ -56,6 +56,23 @@ public static Jbig2Bitmap[] DecodePatternDictionary(
             ref mq, contexts, count * patternWidth, patternHeight, template,
             typicalPrediction: false, CollectiveAt(patternWidth), budget);
 
+        return SlicePatterns(collective, count, patternWidth, patternHeight);
+    }
+
+    /// <summary>
+    /// <see cref="DecodePatternDictionary(ref MqDecoder, int, int, int, int, Jbig2PixelBudget)"/>
+    /// for HDMMR = 1: the collective bitmap is MMR-coded (T.88 §6.7.5, Table 27).
+    /// </summary>
+    public static Jbig2Bitmap[] DecodePatternDictionaryMmr(
+        ReadOnlySpan<byte> data, int patternWidth, int patternHeight, int maxIndex, Jbig2PixelBudget budget)
+    {
+        var count = maxIndex + 1;
+        var collective = MmrDecoder.Decode(data, count * patternWidth, patternHeight, budget);
+        return SlicePatterns(collective, count, patternWidth, patternHeight);
+    }
+
+    private static Jbig2Bitmap[] SlicePatterns(Jbig2Bitmap collective, int count, int patternWidth, int patternHeight)
+    {
         var patterns = new Jbig2Bitmap[count];
         for (var i = 0; i < count; i++)
             patterns[i] = collective.Crop(i * patternWidth, 0, patternWidth, patternHeight);
@@ -75,6 +92,8 @@ public static Jbig2Bitmap[] DecodePatternDictionary(
     /// <param name="Template">HTEMPLATE, the generic template each bitplane uses.</param>
     /// <param name="DefaultPixel">HDEFPIXEL.</param>
     /// <param name="Combination">HCOMBOP.</param>
+    /// <param name="Mmr">HMMR: the grey-scale bitplanes are MMR-coded.</param>
+    /// <param name="EnableSkip">HENABLESKIP: grid cells whose pattern falls wholly outside the region are not coded.</param>
     internal readonly record struct HalftoneParameters(
         int Width,
         int Height,
@@ -86,22 +105,47 @@ internal readonly record struct HalftoneParameters(
         int VectorY,
         int Template,
         byte DefaultPixel,
-        CombinationOperator Combination);
+        CombinationOperator Combination,
+        bool Mmr = false,
+        bool EnableSkip = false);
 
     /// <summary>Decodes a halftone region (T.88 §6.6.5).</summary>
-    /// <param name="mq">The arithmetic decoder, positioned at the grey-scale bitplanes.</param>
+    /// <param name="data">The coded grey-scale bitplanes, after the segment's fixed fields.</param>
     /// <param name="p">The region parameters, read from the segment header.</param>
     /// <param name="patterns">The dither patterns from the pattern dictionary this region names.</param>
     /// <param name="budget">The decode's remaining pixel allowance.</param>
     public static Jbig2Bitmap DecodeHalftoneRegion(
-        ref MqDecoder mq, HalftoneParameters p, Jbig2Bitmap[] patterns, Jbig2PixelBudget budget)
+        ReadOnlySpan<byte> data, HalftoneParameters p, Jbig2Bitmap[] patterns, Jbig2PixelBudget budget)
     {
         if (patterns.Length == 0)
             throw new InvalidDataException("JBIG2: halftone region refers to an empty pattern dictionary.");
 
         budget.Charge(p.Width, p.Height);
         var region = new Jbig2Bitmap(p.Width, p.Height, p.DefaultPixel);
-        var grey = DecodeGrayscale(ref mq, p.GridWidth, p.GridHeight, patterns.Length, p.Template, budget);
+
+        // §6.6.5.1: with HENABLESKIP, a grid cell whose pattern would land wholly
+        // outside the region is not coded in any bitplane. The test is on the
+        // pattern's top-left corner, in whole pixels, against the pattern size.
+        Jbig2Bitmap? skip = null;
+        if (p.EnableSkip)
+        {
+            budget.Charge(p.GridWidth, p.GridHeight);
+            skip = new Jbig2Bitmap(p.GridWidth, p.GridHeight);
+            var patternWidth = patterns[0].Width;
+            var patternHeight = patterns[0].Height;
+            for (var m = 0; m < p.GridHeight; m++)
+            {
+                for (var n = 0; n < p.GridWidth; n++)
+                {
+                    var x = (p.GridX + (long)m * p.VectorY + (long)n * p.VectorX) >> 8;
+                    var y = (p.GridY + (long)m * p.VectorX - (long)n * p.VectorY) >> 8;
+                    if (x + patternWidth <= 0 || x >= p.Width || y + patternHeight <= 0 || y >= p.Height)
+                        skip.Data[m * p.GridWidth + n] = 1;
+                }
+            }
+        }
+
+        var grey = DecodeGrayscale(data, p, patterns.Length, skip, budget);
 
         for (var m = 0; m < p.GridHeight; m++)
         {
@@ -111,15 +155,24 @@ public static Jbig2Bitmap DecodeHalftoneRegion(
                 // a vector, so a dither grid can be rotated or sheared relative to
                 // the page. Coordinates are in 1/256 pixel units throughout, hence
                 // the shift rather than a division.
-                var x = p.GridX + m * p.VectorY + n * p.VectorX;
-                var y = p.GridY + m * p.VectorX - n * p.VectorY;
+                // In long: HGX is a full 32-bit value, and a grid of 2^24 cells times a
+                // 16-bit vector overflows int long before it is implausible.
+                var x = p.GridX + (long)m * p.VectorY + (long)n * p.VectorX;
+                var y = p.GridY + (long)m * p.VectorX - (long)n * p.VectorY;
 
                 var level = grey[m * p.GridWidth + n];
                 if ((uint)level >= (uint)patterns.Length)
                     throw new InvalidDataException(
                         $"JBIG2: halftone grey level {level} exceeds the {patterns.Length}-pattern dictionary.");
 
-                region.Combine(patterns[level], x >> 8, y >> 8, p.Combination);
+                var px = x >> 8;
+                var py = y >> 8;
+                // A pattern wholly off the region changes nothing; skipping it here
+                // also keeps the coordinates inside int for Combine.
+                if (px >= p.Width || py >= p.Height || px + patterns[level].Width <= 0 || py + patterns[level].Height <= 0)
+                    continue;
+
+                region.Combine(patterns[level], (int)px, (int)py, p.Combination);
             }
         }
 
@@ -132,14 +185,21 @@ public static Jbig2Bitmap DecodeHalftoneRegion(
     /// above it to undo the Gray coding.
     /// </summary>
     private static int[] DecodeGrayscale(
-        ref MqDecoder mq, int width, int height, int levels, int template, Jbig2PixelBudget budget)
+        ReadOnlySpan<byte> data, HalftoneParameters p, int levels, Jbig2Bitmap? skip, Jbig2PixelBudget budget)
     {
+        var width = p.GridWidth;
+        var height = p.GridHeight;
+
+        // HBPP = ceil(log2(HNUMPATS)), at least one plane. levels is at most 2^32,
+        // so this stops by 32.
         var planes = 0;
-        while (1 << planes < levels) planes++;
+        while (planes < 32 && 1L << planes < levels) planes++;
         planes = Math.Max(planes, 1);
 
-        var contexts = new byte[1 << GenericRegionDecoder.ContextBits(template)];
-        var at = GrayscaleAt(template);
+        var contexts = new byte[1 << GenericRegionDecoder.ContextBits(p.Template)];
+        var at = GrayscaleAt(p.Template);
+        var mq = new MqDecoder(data);
+        var mmrOffset = 0;
 
         // The caller has already bounded the grid by Jbig2Limits.MaxHalftoneGridCells,
         // which is what keeps this product inside int and these two arrays (five
@@ -149,8 +209,19 @@ private static int[] DecodeGrayscale(
 
         for (var bit = planes - 1; bit >= 0; bit--)
         {
-            var plane = GenericRegionDecoder.Decode(
-                ref mq, contexts, width, height, template, typicalPrediction: false, at, budget);
+            Jbig2Bitmap plane;
+            if (p.Mmr)
+            {
+                // C.5 with GSMMR = 1: each plane is its own MMR bitmap, starting on the
+                // byte after the previous one ended. SKIP does not apply to MMR coding.
+                plane = MmrDecoder.Decode(data[Math.Min(mmrOffset, data.Length)..], width, height, budget, out var used);
+                mmrOffset += used;
+            }
+            else
+            {
+                plane = GenericRegionDecoder.Decode(
+                    ref mq, contexts, width, height, p.Template, typicalPrediction: false, at, budget, skip);
+            }
 
             // C.5: every plane but the first is coded relative to the one above,
             // so the running XOR is what turns Gray code back into binary.