Keep the checksum lint's regression corpus - #9
Merged
Merged
Conversation
The checksum lint took nineteen rounds of review. Every round was a definition shape that slipped past it, and every fix was verified once, by hand, against a scratch file that was then deleted — so the corpus survives only in dfcd595's commit message. Nothing stops the next edit reintroducing any of them. test/lint is that corpus, kept. Each case writes a fixture definition, calls lint_checksums, and asserts on the `failures` array — the same array bin/ci's summary reads, so a test can't pass on output that wouldn't fail the run. Both directions are covered, because they aren't symmetric. A false negative ships an unverified download silently, which is what the lint exists to prevent; a false positive blocks CI over a URL nothing fetches, which is what gets checks deleted. Twenty undigested forms must fail — every quoting style, continuations, split mid-scheme, after if/&&/;/ANSI-C quoting/a multi-line string/a heredoc/a quoted <<HIDE, attached ;echo/&&/|/), ?download=1, uppercase and wrong-length digests. Reference links in comments and the savannah gitweb URLs must pass. And a file with no archives at all must fail rather than pass over nothing. Three things nothing else exercises: the archive extensions (.tar.xz, .tbz2, .tar.Z, .zip are in the pattern but no definition uses them), join_continuations itself (no definition has a line continuation), and the vacuous-pass guard. bin/ci changes only enough to be sourceable: the dispatch moves into main(), called only under `[ "${BASH_SOURCE[0]}" = "$0" ]`. Sourcing it currently cds, prints a banner, runs all three lints over the sourcing script's arguments, and can exit 1 out of the harness. The header comment is deliberately not renumbered — --help re-reads lines 2-15 of the file. bin/ci runs test/lint as well as syntax-checking and shellchecking it. Beyond what the plan called for, and easy to drop, but a regression suite CI never executes rots unnoticed, which is the failure mode this file exists to document. Verified by mutation, since a suite that can't fail is worth nothing: dropping .tar.xz from the pattern, joining continuations with a space, removing the vacuous-pass guard, accepting uppercase digests, and classifying every URL as an archive each fail the cases they should and no others.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The checksum lint took nineteen rounds of review. Every round was a definition shape that
slipped past it, and every fix was verified once, by hand, against a scratch file that was
then deleted — so the corpus survives only in dfcd595's commit message. Nothing stops the
next edit reintroducing any of them.
test/lintis that corpus, kept. Each case writes a fixture definition, callslint_checksums, and asserts on thefailuresarray — the same array bin/ci's summaryreads, so a test can't pass on output that wouldn't fail the run.
What's covered
Both directions, because they aren't symmetric. A false negative ships an unverified
download silently, which is what the lint exists to prevent; a false positive blocks CI
over a URL nothing fetches, which is what gets checks deleted.
after
if/&&/;/ANSI-C quoting/a multi-line string/a heredoc/a quoted<<HIDE,attached
;echo/&&/|/),?download=1, uppercase digest, 67- and 63-hex digests.?a=1&b=2#<digest>, continued and split-schemeURLs that do carry digests.
skipped by shape rather than by an exemption list.
.tar.xz,.tbz2,.tar.Z,.zip,join_continuationsitself, and the vacuous-pass guard.bin/ci changes
Only enough to be sourceable: the dispatch moves into
main(), called under[ "${BASH_SOURCE[0]}" = "$0" ]. Sourcing it currently cds, prints a banner, runs allthree lints over the sourcing script's arguments, and can
exit 1out of the harness. Theheader comment is deliberately not renumbered —
--helpre-reads lines 2-15 of itself.test/lintis added tolint_syntaxandlint_shellcheck, and bin/ci also runs it. Thatlast part is beyond what I'd planned and easy to drop, but a regression suite CI never
executes rots unnoticed, which is the failure mode this file exists to document.
Verification
bin/cigreen with signoff: lint +test/lint35/35 + 12/12 build matrix.--severity=warningonbin/ci,test/build,test/lint..tar.xzfrom the pattern, joining continuations with a space, removing the vacuous-pass guard,
accepting uppercase digests, and classifying every URL as an archive each fail exactly
the cases they should and no others.