Skip to content

Palette mode in the container, encode options (mode/effort), faster decoding - #2

Merged
libphf merged 1 commit into
mainfrom
feat/encode-modes
Oct 28, 2026
Merged

libphf merged 1 commit into
mainfrom
feat/encode-modes

Conversation

@libphf

@libphf libphf commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Summary

Brings the monograph's compression by pixel (the NHF3 palette mode, already in the engine) into the PHF container, lets callers choose between speed and size, and makes decoding faster and lighter. The file format is unchanged except that writers may now emit the palette mode, which this PR documents in SPEC.md.

Encoder options

phf_encode_memory() / phf_write_file() now take const phf_encode_options_t * (NULL or {0} = AUTO, BEST):

PHF_EFFORT_BEST (files) PHF_EFFORT_FAST (streaming)
PHF_MODE_MED both plane variants (plain / run-length), smaller kept one encode per plane
PHF_MODE_PALETTE same, palette stream one encode
PHF_MODE_AUTO tries MED and palette up to 4096 colours, smaller kept palette only up to 256 colours, else MED; no trials

Streaming applications are expected to fix the mode up front (webcam/video → MED, screen content → palette). CLI: --mode=auto|med|palette, --fast; info reports the coding mode.

Decoder (no format change)

  • Bug: palette mode left alpha at 0 when decoding RGB into a 4-byte stride, so palette images would be transparent in the browser. Fixed; covered by tests.
  • Huffman decoders are allocated only for the tables a plane carries (was always 259 × ~2.4 KB = 618 KB per plane; measured need 47–157 KB). Matters for small-memory targets and caches.
  • Inverse MED runs the three colour planes in one interleaved pass when OpenMP does not split them: ~2.1× for that stage, bit-identical.

Measurements

Single thread, Apple M3 Pro, main → this branch:

Image MED+FAST encode / decode AUTO+BEST size
photo 1080p 38.8 → 15.4 ms / 30.9 → 27.2 ms =
camera 720p 15.9 → 10.1 ms / 13.7 → 11.3 ms =
video frame 1080p 35.3 → 22.7 ms / 33.0 → 25.8 ms =
graphics (monograph GIFs) similar −71 % to −76 %, decode 2–3× faster

MED+FAST sizes stay within ±2 % of main. BEST encodes more variants, so it is slower than before (still well below PNG level 9).

Spec

SPEC.md section 4 describes the palette stream byte by byte; flags 0x08/0x09 mark palette files. An independent decoder written only from the text reproduced 12 encoder outputs bit-exactly (RGB/RGBA, 1–300 colours, with and without runs, both efforts).

Breaking changes

  • New options parameter on phf_encode_memory() and phf_write_file(); nhf3_encode() / nhf3_plane_encode() gain an effort argument.
  • apps/web/phf.wasm must be rebuilt for the alpha fix before palette files are served to the web viewer.
  • phf-stream-webcam / phf-stream-video call the old signature; they keep working with their bundled library and will be updated to { MED, FAST } in follow-up PRs.

Test plan

  • 2686/2686 checks (was 470): round-trips in all 6 mode/effort combinations, stride 3/4 with alpha, mode selection, invalid options, palette fuzz, legacy sample.phf.
  • Same suite under ASan + UBSan and in a no-OpenMP Release build (exercises the interleaved inverse MED at every size).
  • CLI: auto picks palette for a graphic and MED for a camera frame; decode bit-exact.
  • CI (Linux OpenMP + sanitizers, macOS, Windows MSVC).
  • WASM rebuild and browser check of a palette file.

🤖 Generated with Claude Code

Encoder:
- phf_encode_memory() and phf_write_file() take phf_encode_options_t
  { mode, effort } (NULL or {0} = AUTO, BEST).
  mode: MED (photos, camera, video), PALETTE (graphics, few colours: the
  monograph's compression by pixel, already in the NHF3 engine), AUTO.
  effort: BEST tries MED and palette up to 4096 colours and both plane
  variants (plain/run-length), keeping the smaller; FAST encodes once,
  for streaming (AUTO uses palette only up to 256 colours).
- FAST picks the run-length variant only from 60% repeated symbols: the
  discarded second pass was 40-65% of encode time on photos and camera.
- Palette files carry flags 0x08/0x09 (no MED/DECORR bits).

Decoder (format unchanged):
- Fix palette mode leaving alpha at 0 when decoding RGB into a 4-byte
  stride (images were transparent in the browser).
- Allocate Huffman decoders only for the tables a plane carries instead of
  all 259 (618 KB -> typically 50-160 KB per plane).
- Inverse MED runs the three colour planes in one interleaved pass when
  OpenMP does not split them (about 2x for that stage).

Measured single-threaded on Apple M3 Pro, main -> this branch:
- MED+FAST: 1080p photo encode 38.8 -> 15.4 ms, decode 30.9 -> 27.2 ms;
  sizes within +-2%.
- AUTO+BEST on graphics: 71-76% smaller, 2-3x faster to decode.

Also: SPEC.md section 4 documents the palette mode byte by byte
(validated with an independent decoder written from the text only), CLI
gains --mode=auto|med|palette and --fast and reports the coding mode,
bench.py and README follow the new API, and the test suite grows from 470
to 2686 checks (every mode and effort, mode selection, palette fuzz).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@libphf
libphf merged commit 888d273 into main Oct 28, 2026
3 of 4 checks passed
@libphf
libphf deleted the feat/encode-modes branch October 28, 2026 19:14
libphf added a commit that referenced this pull request Oct 28, 2026
Palette mode in the container, encode options (mode/effort), faster decoding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants