Repository navigation
perf(deflate,checksum): faster deflate encoder and checksums - #166
Merged
Merged
Conversation
Adler-32 now sums each 16-byte block across sixteen independent lanes (auto-vectorised) and folds them once per NMAX chunk: ~7.7 GB/s, about twice the old byte loop, on Apple M5. CRC-32 moves from slice-by-8 to slice-by-16 (~4.3 GB/s, up from ~2.8). Both are cross-checked against bytewise references. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Multiplicative 3-byte hash; insert returns the old chain head, so the match search starts from it without rehashing. - Chain walk rejects candidates on the four bytes ending at the current best length, searches only for matches longer than the pending lazy one, and folds NIL/stale/too-far into one distance check. - zlib's max_lazy cut-off at levels 6-7 (good 8, lazy 32). - Symbols packed into u32 and histograms tallied during the LZ77 pass; the stored fallback reads the window instead of a per-block copy. - Moffat-Katajainen in-place Huffman lengths, falling back to package-merge only when a code would exceed the limit. - Branch-free 64-bit bit writer over a pre-sized buffer; length and distance codes plus extra bits go out in one write. Sizes move by at most +0.1% at any level on text, binaries, repetitive and random input; most shrink (level 1 by up to 2.6%). The shared crate::bits::BitWriter is now only built for deflate64. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 31, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Encoder speed item of #156. The deflate encoder is now 1.1–2.8× zlib's speed at levels 1/6/9 on text, binaries, repetitive and random data (before: 0.6–0.95×). Output size changes by at most +0.1% at any level 1–9, and most levels shrink (level 1 by up to 2.6%).
Measured on Apple M5 Max, raw deflate, against system zlib 1.2.12; full table in BENCH.md.
Changes
No public API changes. All tests pass, including the Python zlib cross-checks for every strategy and flush mode.
🤖 Generated with Claude Code