chore(rules): collect third-party rule files recursively before running fixup_rules - #1703
Merged
Merged
Conversation
…ng `fixup_rules` Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
egibs
enabled auto-merge (squash)
September 8, 2026 13:13
mattlorimor
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Collect third-party rule files recursively before running
fixup_rules, and makefixup_rulesfail when it receives no files.third_party/yara/update.sh:254-261replaces the top-level glob"${kind}"/*.yar*with afind -print0 | sort -zpipeline read into an array.third_party/yara/update.sh:78-82returns 1 with a message whenfixup_rulesis called without arguments. A bareperl -iwould otherwise block reading stdin.bartblaze/,JPCERT/, andTTC-CERT/change by trailing-whitespace removal only, the result of running the fixed script.RELEASEandSHA256SUMSfiles are unchanged.Why
The scheduled third-party update failed on its first run after #1684 landed
hack/literal_regexps.pl:https://github.com/chainguard-dev/malcontent/actions/runs/34224542383/job/102055349946
Three sources keep rules in subdirectories. bartblaze and TTC-CERT have no top-level
.yar*files, and JPCERT has 75 nested ones alongside 87 at the top level. For TTC-CERT the glob matched nothing, so bash passed the literal pattern through. The inlineperl -icalls warn and exit 0 on an unopenable path, which hid the problem. The new script dies instead, andset -eaborts the run. bartblaze would have failed next in the same run.The consequence is that
fixup_rulesnever ran on bartblaze or TTC-CERT, or on JPCERT's nested files. The whitespace-only diff in this PR is the first application of those fixups.literal_regexps.plfound nothing to rewrite in these sources.Notes
/bin/bashshebang, which is bash 3.2 on macOS, so the fix avoidsglobstarandmapfile -d.findutilsbut notcoreutils, sosortthere is BusyBox. Thefind -print0 | sort -z | read -d ''pipeline was verified in the pinnedwolfi-baseimage from.github/workflows/third-party.yaml:23../third_party/yara/update.shfor TTC-CERT, bartblaze, and JPCERT exits 0 and passes 22, 111, and 162 files toliteral_regexps.pl.fixup_ruleswith no arguments returns 1 instead of hanging.bash -nandshellcheck -S warningare clean.yr compile --path-as-namespace(yara-x 1.20.0) succeeds on the three changed sources.go test ./pkg/compile/ -run TestRecursivepasses against a rebuilt yara-x 1.20.0 C library, compiling every embedded rule set including the changed files.