Skip to content

chore(flare): drop mholt/archiver for stdlib archive/zip - #3363

Open
wdhif wants to merge 6 commits into
mainfrom
claude/archives-library-upgrade-38d5ab
Open

chore(flare): drop mholt/archiver for stdlib archive/zip#3363
wdhif wants to merge 6 commits into
mainfrom
claude/archives-library-upgrade-38d5ab

Conversation

@wdhif

@wdhif wdhif commented Aug 17, 2026

Copy link
Copy Markdown
Member

What does this PR do?

Replaces the single github.com/mholt/archiver/v3 call site in kubectl datadog flare
with the standard library's archive/zip, and drops the dependency.

  • cmd/kubectl-datadog/flare/flare.go: new archiveDir() helper replaces
    archiver.NewZip().Archive(); the zip *archiver.Zip field on options is gone.
  • cmd/kubectl-datadog/flare/archive_test.go: new unit test pinning the archive layout
    (top-level folder, nested dirs, non-regular files skipped).
  • go.mod / go.sum / LICENSE-3rdparty.csv: removes 1 direct + 9 indirect modules.

No change to the archive the flare command produces. The one observable difference is
the error text when two flares collide on the same filename — see Additional Notes.

Motivation

mholt/archiver/v3 is deprecated and frozen at v3.5.1, which carries two path traversal
advisories:

advisory severity vector
CVE-2025-3445 High (8.1) Zip Slip via a crafted zip
CVE-2024-0406 Moderate (6.1) path traversal via a crafted tar

Both are reachable only through archiver.Unarchive, and neither can be cleared by a
version bump: no fix was ever published — v3.5.1 is the last version on the module proxy,
so the 3.5.2 that some scanners cite as the patched release does not exist — and
upstream's only remediation is the rename to mholt/archives.

The flare command only ever wrote a zip of a single directory, so the vulnerable
extraction path was never reachable from the operator, but the dependency still shows up in
dependency scans with no remediation available in place.

Migrating to mholt/archives was the other option and was rejected: it is pre-1.0
(v0.1.5, no API stability commitment) and it grows the footprint, requiring ~14 modules
(adding bodgit/sevenzip, STARRY-S/zip, minio/minlz, sorairolake/lzip-go,
mikelolasagasti/xz, spf13/afero, hashicorp/golang-lru) — i.e. pulling 7z/rar/lzip
decoders into the operator in order to write one zip.

Going to the standard library instead removes 10 modules for ~60 lines of code:

before after
modules attributable to this dep 10 0
LICENSE-3rdparty.csv rows 9 0
kubectl-datadog binary 163.0 MB 161.1 MB

The removed set includes github.com/xi2/xz, which we currently ship with its license
listed as Unknown.

Additional Notes

Output equivalence. Zipping one real flare directory with both implementations produces
identical entry names and order, directory entries, compression methods, CRC32s,
uncompressed sizes, decompressed bytes, flag bits, creator/extract versions, permissions
(0755 dir / 0644 files), DOS timestamps and extended-timestamp extra fields.

The one observable difference is compressed sizearchiver deflated through
github.com/klauspost/compress/zip rather than archive/zip, so the same input now yields
a slightly smaller archive:

entry archiver v3 stdlib
datadog-custom-resources.yaml 2610 2385
<pod>-metrics.txt 3899 3712
<pod>-status.txt 749 735
<pod>.json 2593 2501
total 9851 9333 (−5.3%)

Same DEFLATE format, so no consumer is affected.

Same-second collisions behave as before, with different wording. getArchivePath() only
has second precision, so two flares finishing collection in the same second pick the same
name. archiver.NewZip() refused to write over an existing archive
(if !z.OverwriteExisting && fileExists(dest)), and archiveDir keeps that guarantee by
opening the destination with O_EXCL — which is additionally atomic, where archiver's
check-then-os.Create had a TOCTOU window in which both runs could truncate. The command
still fails in that case, only the message changed:

before: file already exists: /tmp/datadog-operator-2026-08-25-14-43-16.zip
after:  open /tmp/datadog-operator-2026-08-25-14-43-16.zip: file exists

A failed archive is also removed rather than left truncated on disk, since with O_EXCL a
leftover would block the next flare of that second. Note this only serialises the archive
step — baseDir is a fixed path, so concurrent flares already interleave their collected
files; making concurrent runs genuinely safe needs a per-run directory and is out of scope
here.

One latent behaviour difference: archiver had SelectiveCompression, which stored
files whose extension is already compressed (.gz, .png, .zip, …); archiveDir always
deflates. The flare only ever writes .yaml, .txt and .json, so this is unreachable
today — replicating it would mean copying a 30-entry extension map for no gain.

go.work.sum picks up 3 /go.mod hashes (brotli, snappy, pierrec/lz4) that moved
out of go.sum when the modules left the main module's graph. Stable across make sync +
rebuild.

Minimum Agent Versions

None — the change is confined to the kubectl plugin.

  • Agent: n/a
  • Cluster Agent: n/a

Describe your test plan

Automated: make lint, go build ./..., go test ./... (106 packages ok, 0 failures),
make verify-licenses (exit 0, no diff).

Manual, on minikube (minikube start, Kubernetes v1.35.1) with operator 1.28.0 installed
from the datadog-operator Helm chart plus a DatadogAgent CR:

  1. Build the plugin before and after the change:
    go build -o /tmp/flare-before ./cmd/kubectl-datadog/main.go on main, same for this
    branch. go version -m on each confirms github.com/mholt/archiver/v3 is linked into
    the first and absent from the second.
  2. Run each with an isolated TMPDIR so the artifacts don't collide:
    TMPDIR=/tmp/before; echo n | /tmp/flare-before flare --email <you> --apiKey <key>.
    Answer n at the upload prompt — the flare is written locally and nothing is sent to
    Datadog.
  3. Compare the two archives: same entries, order, directory entry, compression methods and
    permissions. The collected -metrics.txt and .json payloads differ only because they
    carry live counter and log data; the stable files (datadog-custom-resources.yaml,
    -status.txt) come out with identical CRCs and content hashes.
  4. For a controlled comparison, zip the same directory with both implementations, then
    unzip both and diff -r — extracted trees are byte-identical, verified per file with
    shasum -a 256. Every zip header field matches except compress_size.
  5. unzip -t <flare>.zip reports no errors on the new archive, and the collected
    datadog-custom-resources.yaml still has the API key redacted.

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring,
    documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

@wdhif wdhif added enhancement New feature or request qa/skip-qa and removed qa/skip-qa labels Aug 17, 2026
@wdhif wdhif added this to the v1.31.0 milestone Aug 17, 2026
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 17, 2026

Copy link
Copy Markdown

Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 82.50%
Overall Coverage: 50.22% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a2b0b16 | Docs | View more details | Give us feedback!

@wdhif

wdhif commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

/dd-review codex

@wdhif
wdhif force-pushed the claude/archives-library-upgrade-38d5ab branch from 6e9ce53 to 4209b87 Compare August 19, 2026 14:15
@wdhif
wdhif force-pushed the claude/archives-library-upgrade-38d5ab branch from 4209b87 to d05d4a8 Compare August 24, 2026 11:51
@wdhif wdhif added refactoring dependencies Pull requests that update a dependency file qa/skip-qa labels Aug 25, 2026
@wdhif
wdhif marked this pull request as ready for review August 25, 2026 12:52
@wdhif
wdhif requested review from a team and a lite review from Copilot August 25, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the kubectl datadog flare implementation to stop using the deprecated github.com/mholt/archiver/v3 dependency and instead create flare zip archives using Go’s standard library (archive/zip). This reduces the dependency footprint and avoids known advisories associated with the removed module.

Changes:

  • Replaced the archiver.NewZip().Archive() call site with a new archiveDir() helper built on archive/zip.
  • Added unit tests that pin the zip layout (top-level folder prefix, nested directories, and skipping non-regular files like symlinks).
  • Removed mholt/archiver/v3 and related transitive dependencies from go.mod/go.sum, and updated third-party license tracking.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
LICENSE-3rdparty.csv Removes third-party license rows associated with the dropped archiver dependency tree.
go.work.sum Updates workspace sum entries reflecting module graph changes after dependency removal.
go.sum Removes checksums for archiver-related modules no longer in the main module graph.
go.mod Drops github.com/mholt/archiver/v3 and related indirect dependencies.
cmd/kubectl-datadog/flare/flare.go Introduces archiveDir() using archive/zip and removes the archiver.Zip field/call site.
cmd/kubectl-datadog/flare/archive_test.go Adds unit tests verifying archive layout and behavior when the destination file already exists.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cmd/kubectl-datadog/flare/flare.go
Comment thread cmd/kubectl-datadog/flare/flare.go
@wdhif

wdhif commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

/dd-review codex

wdhif and others added 5 commits August 25, 2026 16:42
github.com/mholt/archiver/v3 is deprecated and frozen at v3.5.1, which
carries two unfixable path traversal advisories: CVE-2025-3445 (Zip Slip via
a crafted zip) and CVE-2024-0406 (path traversal via a crafted tar), both
reachable only through archiver.Unarchive. No fix was ever published --
v3.5.1 is the last version on the module proxy -- and upstream's only
remediation is the rename to mholt/archives, so neither finding can be
cleared by a version bump.

The flare command only ever wrote a zip of a single directory, so the
vulnerable extraction path was never reachable, but the dependency still
surfaces in dependency scans and pulls 9 modules into the operator for one
call site -- among them github.com/xi2/xz, which we ship with an
unidentified license. Migrating to mholt/archives would instead grow that
set to ~14 modules on a pre-1.0 API, so the single call is replaced with
archive/zip.

Zipping one real flare directory with both implementations yields identical
entry names and order, directory entries, methods, CRCs, permissions,
timestamps and extracted bytes. The only difference is compressed size:
archiver deflated through github.com/klauspost/compress, so the same input
now produces a ~5% smaller archive (9333 vs 9851 bytes). The plugin binary
shrinks by ~1.9 MB.

Co-Authored-By: Claude <noreply@anthropic.com>
The map literal in TestArchiveDir was hand-aligned; gofmt un-aligns the
short key, which made check_formatting fail on git diff --exit-code.

Co-Authored-By: Claude <noreply@anthropic.com>
archiveDir opened the destination with O_TRUNC, while archiver.NewZip()
refused to write over an existing file. getArchivePath only has second
precision, so two flares finishing collection in the same second picked the
same name: instead of the second one failing, both wrote into one file,
producing a corrupt archive or making a command upload the other's artifact.

Open the destination with O_EXCL to restore the previous guarantee. The
shared createFile keeps O_TRUNC, which is what the collected files in the
flare directory need since it persists between runs.

Co-Authored-By: Claude <noreply@anthropic.com>
Also delete the destination when archiving fails: the zip writer has
already created it, so a failure left a truncated archive behind that both
misleads whoever finds it and blocks the next flare of the same second
through O_EXCL. The pre-existing-file case is untouched, since that open
fails before anything is created.

Raises patch coverage on the changed lines from 80.0% to 85.0%, above the
80% gate.

Co-Authored-By: Claude <noreply@anthropic.com>
The os.Open branch is the last reachable uncovered path in archiveDir; the
remaining ones are the call site in run(), which has no test harness, and
errors from filepath.Rel, zip.FileInfoHeader and CreateHeader, which cannot
be triggered on paths that filepath.Walk just produced.

Skipped under root, which reads files whatever their permission bits say.

Co-Authored-By: Claude <noreply@anthropic.com>
@wdhif
wdhif force-pushed the claude/archives-library-upgrade-38d5ab branch from 3003ac6 to e7e1f64 Compare August 26, 2026 09:33
The symlink case lived in the layout test, so on a Windows host without
Developer Mode that test failed in setup, reporting a layout problem that
was really a missing privilege. It now has its own test that skips there.

The unreadable-file case skips on Windows too: permission bits do not stop
a read there, and Geteuid returns -1 so the root check never fired, leaving
both assertions to fail rather than skip.

Neither affects CI, which is Linux only, but the kubectl plugin ships a
Windows binary so contributors may well run the suite there. Coverage on
Linux is unchanged, the same branches still execute.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file enhancement New feature or request qa/skip-qa refactoring team/container-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants