Extend comment scanner with piped-stdin, runner-string, MDX, and worktree TypeScript support

.github/workflows/ci.yml

@@ -896,6 +896,25 @@ jobs:
       - name: Install dependencies
         run: uv sync --group dev
 
+      - name: Set up Node.js for TypeScript comment parsing
+        uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444
+        with:
+          node-version: '20'
+
+      - name: Install locked TypeScript parser
+        shell: bash
+        run: |
+          set -euo pipefail
+          package_dir="$RUNNER_TEMP/comment-policy-node"
+          mkdir -p "$package_dir"
+          cp quality/comment-policy-package.json "$package_dir/package.json"
+          cp quality/comment-policy-package-lock.json "$package_dir/package-lock.json"
+          (
+            cd "$package_dir"
+            npm ci --ignore-scripts --no-audit --no-fund
+          )
+          echo "COMMENT_POLICY_TYPESCRIPT=$package_dir/node_modules/typescript" >> "$GITHUB_ENV"
+
       - name: Download medium collection evidence
         uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
         with:

docs/internal/comment-policy.md

@@ -144,6 +144,18 @@ Python is read with its AST and tokenizer, including standalone strings and
 runtime docstring assignments; JavaScript and TypeScript with the isolated
 TypeScript parser; SQL-valued strings and execution-sink arguments as SQL; other
 formats with Pygments lexers; embedded code with its own language check.
+Shell heredocs and literal `echo`/`printf` pipelines through plain `cat` stages
+or supported `sudo`, `env` and `nohup` wrappers into supported stdin
+interpreters are scanned. Static `sudo`/`env` producer wrappers and static SSH
+pipelines into supported remote stdin interpreters are also scanned. SQL-client
+stdin for `psql`, `duckdb` and `sqlite3` is scanned when the command reads from
+stdin; explicit query/file inputs and `sqlite3 -no-stdin` are not treated as
+piped input. `watch` and `xargs` pipelines to stdin interpreters fail closed
+when input routing is ambiguous. MDX top-level ESM imports and exports,
+including TypeScript `type`, `interface`, `enum`, `namespace`, `declare` and
+`abstract` exports, and JSX block comments outside Markdown code spans and
+fences are extracted. Executable paths are matched by basename; consumer
+output redirects preserve piped input, while redirected stdin fails closed.
 Unknown input is a coverage error, never a pass, and under blocking enforcement
 or in strict mode changed files and completed scopes always reject it. While
 enforcement is advisory a coverage error is reported and counted but does not
@@ -172,8 +184,12 @@ Known gaps, each a place where a comment can pass unreported:
 - Unknown programs that take `-c` or `-e` as data are listed in
   `DATA_FLAG_PROGRAMS` in `scripts/comment_execution.py`; any other program
   given such a flag is reported.
-- Commands passed to a runner such as `ssh` or `watch` as one string are not
-  split, so an interpreter inside that string is not followed.
+- Static `sudo`, `env` and `nohup` interpreter wrappers and `sudo`/`env`
+  literal-producer wrappers with supported options are scanned, including
+  nested static wrapper chains; dynamic or unsupported wrapper options fail
+  closed. Static inner shell commands passed as one argument to runners such as
+  `ssh`, `watch` or `xargs` are followed; dynamically assembled commands are
+  not.
 - HTML comments and MyST `%` lines in Markdown prose are not scanned. In the
   maintained docs they are copyright headers, generator start and end markers,
   `<!-- content-ok -->` markers read by `scripts/blog_content_validation.py`,
@@ -183,10 +199,12 @@ Known gaps, each a place where a comment can pass unreported:
   such comments where they are part of the product's output, for example
   maintenance SQL headers, dry-run DDL previews, stream-file headers, the Trino
   tuning note and the AWS Glue job script.
-- A program piped to an unmodeled interpreter on stdin (such as `printf '...' | perl`
-  or `echo '...' | ruby`) is not scanned.
-- A whole-file pinned notice in `quality/comment-cleanup-scope.json` (such as
-  `_sources/tpc-h/PATCHES.md`) is verified against its recorded digest, but
-  changes to the file are not guarded to require an accompanying pin update on branches.
-- MDX (`.mdx`) files map to the Markdown scanner, so MDX imports and JSX `{/* */}`
-  comments outside Markdown code fences are not inspected.
+- A program piped to an interpreter on stdin from a producer other than `echo`
+  or `printf` (such as `curl ... | sh`) is not scanned.
+- Whole-file pinned notices in `quality/comment-cleanup-scope.json` are checked
+  against their recorded digest, but changed files do not yet require a pin
+  update on the same branch.
+- Unsupported `printf` formats or `echo -e` escapes in a recognized pipeline
+  produce a coverage error.
+- JSX `{// ...}` line-comment expressions in MDX prose are not extracted, so
+  comments in that form outside Markdown code fences are not inspected.

docs/operations/ci-local-parity.md

@@ -154,6 +154,10 @@ merge-gating workflows. Each such step must name a local equivalent or carry
 a written reason in the test's `MERGE_GATE_EXEMPTIONS` table, so a new check
 step cannot become a silent CI-only failure.
 
+The `medium-test` job installs the locked TypeScript parser used by
+`tests/unit/scripts/test_comment_policy.py`; `make test-medium` runs the same
+reviewed-JavaScript flow test locally.
+
 The `code-test` step that runs changed tests on the curated release tree has a
 local equivalent:
 `uv run -- python scripts/release_curation_dry_run.py --changed-since origin/develop`.

scripts/comment_execution.py

@@ -5,10 +5,12 @@
 from collections import defaultdict
 
 from comment_payloads import (
+    RUNNER_COMMANDS,
     SQL_CLIENTS,
     command_words,
     inline_source_index,
     runner_command_start,
+    runner_trailing_index,
     sql_client_source_index,
 )
 
@@ -645,15 +647,23 @@ def process_payload(self, args: list[ast.expr]) -> tuple[ast.AST, str, str | Non
     def runner_payload(
         self, args: list[ast.expr], words: list[str | None], name: str
     ) -> tuple[ast.AST, str, str | None] | None:
+        if name in RUNNER_COMMANDS:
+            inner = self.runner_string_args(args, words, name)
+            if inner is not None:
+                return inner
         try:
             first = runner_command_start(words, name)
         except ValueError:
             return args[0], "unsupported", None
         for index, word in enumerate(words[first:], start=first):
             nested = word.rsplit("/", 1)[-1] if word else None
-            if nested and (
-                nested.startswith("python")
-                or nested in {"node", "sh", "bash", "zsh"} | SQL_CLIENTS | UNMODELED_INTERPRETERS
+            if (
+                nested
+                and not any(char.isspace() for char in nested)
+                and (
+                    nested.startswith("python")
+                    or nested in {"node", "sh", "bash", "zsh"} | SQL_CLIENTS | UNMODELED_INTERPRETERS
+                )
             ):
                 return self.process_payload(args[index:])
         if name not in DATA_FLAG_PROGRAMS and any(
@@ -662,6 +672,18 @@ def runner_payload(
             return args[0], "unsupported", None
         return None
 
+    def runner_string_args(
+        self, args: list[ast.expr], words: list[str | None], name: str
+    ) -> tuple[ast.AST, str, str | None] | None:
+        first = runner_trailing_index(words, name)
+        trailing = words[first:]
+        if len(trailing) != 1 or trailing[0] is None:
+            return None
+        text = self.literal(args[first])
+        if text is None or not text.strip():
+            return None
+        return args[first], "bash", text
+
     def inline_process_payload(
         self, args: list[ast.expr], words: list[str | None], language: str | None
     ) -> tuple[ast.AST, str, str | None] | None: