Skip to content

Verify WOF savings and add thorough analysis - #15

Open
TheKillerMax wants to merge 1 commit into
wefalltomorrow:masterfrom
TheKillerMax:improve/verified-wof-savings
Open

TheKillerMax wants to merge 1 commit into
wefalltomorrow:masterfrom
TheKillerMax:improve/verified-wof-savings

Conversation

@TheKillerMax

@TheKillerMax TheKillerMax commented Oct 9, 2026 •

Copy link
Copy Markdown

Why

The generic compression sample can reject files that the selected WOF codec
would compress, while a favourable estimate does not prove that a real
transformation saved useful disk space. The current workflow also needs
stronger handling of stale analyses, file identity, failed decompression and
operation outcomes before doing more exhaustive work.

Changes

  • Add optional Thorough analysis. Try the selected codec on every otherwise
    eligible, uncached file; keep Sampled as the backwards-compatible default
    and preserve LZX as the existing default codec.
  • Retain new compression only if measured allocation reduction meets the
    configured percentage and saves at least one NTFS cluster. Verify the backing
    state and logical length; attempt restoration below the threshold and report
    measurement or restoration failures explicitly.
  • Inspect actual WOF provider/version/algorithm information and measure through
    a guarded normal file view. Protect file identity, exact canonical path,
    hardlinks and incompatible attributes; coordinate cooperating Compactor
    instances through a per-identity mutex.
  • Keep existing WOF files unchanged during Compress. Cache only measured
    negative results, keyed by protected file metadata, codec, analysis mode,
    threshold and cluster size. Preserve pending cache entries across I/O errors.
  • Require reanalysis after settings changes, decompression or errors. Make
    DirectStorage preflight cancellable and independent of compression
    exclusions. Stop scheduling when the UI closes and try each original
    decompression candidate at most once per run.
  • Validate configuration, save it atomically, escape literal folder exclusions,
    fix progress/viewer state and keep a persistent report with separate changed,
    below-threshold, skipped and failed counts.
  • Remove unsupported codec-specific estimate multipliers. Update the
    crossbeam channel and utility dependencies, support real MinGW cross-builds
    and add documentation plus explicit Windows/NTFS test gates.

Validation completed during preparation

  • 19 tests of the actual embedded JavaScript using a minimal DOM adapter.
  • 8 hashfilter persistence and failure-recovery tests.
  • 6 production policy tests and 2 production background-control tests executed
    on Linux through a standalone harness.
  • cargo fmt --all -- --check, JavaScript syntax and git diff --check.
  • cargo check --locked --offline --target x86_64-pc-windows-gnu --tests.
  • cargo test --workspace --no-run --locked --offline --target x86_64-pc-windows-gnu:
    all workspace test executables compiled and linked with GCC/MinGW.
  • The complete patch applies to a fresh extraction of the supplied source
    archive and reproduces the delivered source tree byte for byte.

Required before merging

  • Windows MSVC release build and ordinary workspace tests pass.
  • Run the three explicit native WOF/NTFS fixtures with
    cargo test --release --locked --target x86_64-pc-windows-msvc -- --ignored --test-threads=1.
  • Complete the relevant native GUI and representative-workload checks in
    docs/TESTING.md.

The included CI and release workflows execute the automated Windows gates.
The Windows x64 workflow run
for this PR is awaiting approval from a repository maintainer. GitHub has
not started any jobs, so the MSVC build and native WOF/NTFS tests remain pending.
Cross-build success and the portable UI adapter do not establish native WOF
or MSHTML behaviour.

Behaviour and tradeoffs

LZX + Thorough offers maximum candidate coverage with that codec; it does
not introduce a fifth WOF format, try all codecs in sequence, or promise a
fixed savings improvement. It can require substantially more CPU time and
writes. Existing compressed files are preserved. Restoring a low-benefit file
requires free space; its preflight is a snapshot rather than a reservation.
DirectStorage detection remains a documented heuristic.

See docs/compression-policy.md for the precise retention and protection
contract and its Microsoft API references. The package version remains
0.11.5; changes are recorded under Unreleased.

Source baseline

Prepared against wefalltomorrow/Compactor:master at 1f415a2b043740e900484411337779a6c173cfb5 (v0.11.5). The current upstream Git tree matches the supplied source archive exactly. The published commit tree was verified against the complete locally validated source tree before opening this PR.

Retain compression only when measured allocation savings meet policy.
Protect file identity and scope, fix cancellation and cache persistence,
and add explicit Windows/NTFS integration coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant