Skip to content

chore(rules): collect third-party rule files recursively before running fixup_rules - #1703

Merged
egibs merged 1 commit into
chainguard-dev:mainfrom
egibs:fix-third-party-update
Sep 8, 2026
Merged

chore(rules): collect third-party rule files recursively before running fixup_rules#1703
egibs merged 1 commit into
chainguard-dev:mainfrom
egibs:fix-third-party-update

Conversation

@egibs

@egibs egibs commented Sep 8, 2026

Copy link
Copy Markdown
Member

What

Collect third-party rule files recursively before running fixup_rules, and make fixup_rules fail when it receives no files.

  • third_party/yara/update.sh:254-261 replaces the top-level glob "${kind}"/*.yar* with a find -print0 | sort -z pipeline read into an array.
  • third_party/yara/update.sh:78-82 returns 1 with a message when fixup_rules is called without arguments. A bare perl -i would otherwise block reading stdin.
  • 58 rule files under bartblaze/, JPCERT/, and TTC-CERT/ change by trailing-whitespace removal only, the result of running the fixed script. RELEASE and SHA256SUMS files are unchanged.

Why

The scheduled third-party update failed on its first run after #1684 landed hack/literal_regexps.pl:
https://github.com/chainguard-dev/malcontent/actions/runs/34224542383/job/102055349946

+ fixup_rules 'TTC-CERT/*.yar*'
+ perl -p -i -e 's/ +$//;' 'TTC-CERT/*.yar*'
Can't open TTC-CERT/*.yar*: No such file or directory.
+ perl /__w/malcontent/malcontent/hack/literal_regexps.pl 'TTC-CERT/*.yar*'
open TTC-CERT/*.yar*: No such file or directory at /__w/malcontent/malcontent/hack/literal_regexps.pl line 108.
make: *** [Makefile:347: update-third-party] Error 2

Three sources keep rules in subdirectories. bartblaze and TTC-CERT have no top-level .yar* files, and JPCERT has 75 nested ones alongside 87 at the top level. For TTC-CERT the glob matched nothing, so bash passed the literal pattern through. The inline perl -i calls warn and exit 0 on an unopenable path, which hid the problem. The new script dies instead, and set -e aborts the run. bartblaze would have failed next in the same run.

The consequence is that fixup_rules never ran on bartblaze or TTC-CERT, or on JPCERT's nested files. The whitespace-only diff in this PR is the first application of those fixups. literal_regexps.pl found nothing to rewrite in these sources.

Notes

  • The script keeps its /bin/bash shebang, which is bash 3.2 on macOS, so the fix avoids globstar and mapfile -d.
  • The workflow container installs findutils but not coreutils, so sort there is BusyBox. The find -print0 | sort -z | read -d '' pipeline was verified in the pinned wolfi-base image from .github/workflows/third-party.yaml:23.
  • Verification:
    • ./third_party/yara/update.sh for TTC-CERT, bartblaze, and JPCERT exits 0 and passes 22, 111, and 162 files to literal_regexps.pl.
    • fixup_rules with no arguments returns 1 instead of hanging.
    • bash -n and shellcheck -S warning are clean.
    • yr compile --path-as-namespace (yara-x 1.20.0) succeeds on the three changed sources.
    • go test ./pkg/compile/ -run TestRecursive passes against a rebuilt yara-x 1.20.0 C library, compiling every embedded rule set including the changed files.
  • The next scheduled third-party run should produce an upstream-drift-only diff.

…ng `fixup_rules`

Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
@egibs
egibs requested a review from mattlorimor September 8, 2026 13:10
@egibs
egibs enabled auto-merge (squash) September 8, 2026 13:13
@egibs
egibs merged commit e7f4b9d into chainguard-dev:main Sep 8, 2026
17 checks passed
@egibs
egibs deleted the fix-third-party-update branch September 8, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants