Skip to content

fix(security-triage): preserve the kept tracker in dedupe handoffs - #40

Draft
MakerFlyFly wants to merge 5 commits into
audit/m-060-basefrom
fix/security-triage-dedupe-direction
Draft

MakerFlyFly wants to merge 5 commits into
audit/m-060-basefrom
fix/security-triage-dedupe-direction

Conversation

@MakerFlyFly

@MakerFlyFly MakerFlyFly commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • PROBABLE-DUP triage named a tracker to keep but passed the newly triaged tracker first to deduplicate. The receiving selector keeps its first argument, so an approved merge could close the tracker triage said to keep.
  • Emit <kept> <new> from both the proposal and recap, align the skill documentation, and regenerate the packaged runtime copies. Four pytest handoff cases feed the emitted commands through the real deduplicate CLI selector with either numeric order.
  • Keep the composition eval bounded to numeric / #N arguments separated by whitespace, using the selector's case-insensitive alias grammar. Argument direction is checked; plugin dispatch identity and other input normalization are outside these predicates' scope.

Type of change

  • Skill documentation and matching eval coverage
  • Python package
  • Documentation

Test plan

  • Four production handoff cases failed on the base renderers and passed after the direction fix.
  • Production validation at 20ae661d: Linux security-management 839 passed; Windows triage and deduplicate suites 98 passed. Production code and unit tests have not changed since that revision, so these results are retained.
  • Retained complete Linux skill-evals validation (da7b1284): 248 passed, including tests/test_pr_triage_eval.py.
  • Retained targeted assertion validation (da7b1284, Windows): 770 valid forms accepted, 2,314 invalid or mixed-direction forms rejected, including 1,536 mixed-direction forms. Command tokens and argument order were checked against the actual selector.
  • Retained response replay (da7b1284): 24/24 saved responses replayed, with 0 new model invocations. Original author-model prose judgements were reused; this is not an independent cross-model review.
  • Related prek hooks and commit hooks passed; Magpie pre-first-pr-check passed. The initial direction fix also passed whole-tree non-test hooks, workspace tests, ruff, format, mypy and lychee.
  • Documentation-only follow-up: 9 synthetic boundary cases were checked with the actual runner and selector, with no model calls. Production code, assertions, expected data and the model prompt retain their blobs; existing test records above are reused. README-related hooks were run for this follow-up.

Complete Windows suites are not claimed as passing. An earlier security-management run returned 794 passed / 45 failed. Complete skill-evals collection on independent base and head worktrees failed with OSError: [Errno 9] Bad file descriptor while reading test_pr_triage_eval.py; subsequent remaining-test diagnostics had the same 46 failed / 195 passed on both revisions. The file was absent during those diagnostics, so they are not full-suite results. The original worktree's file has now been restored from HEAD and verified through Git and native file reads against blob c0ad8a36ba9b737a59f01e78a491c0f51ee0edb8; restoration creates no commit diff. The underlying Windows/Python I/O cause remains unresolved, and Windows infrastructure is outside this change.

RFC-AI-0004 compliance

  • HITL — existing merge confirmation remains required.
  • Vendor neutrality — generic skill parameters and synthetic regression data.
  • Write-access discipline — rendering and selection code execute no tracker mutations.

Linked issues

Refs MakerFlyFly/magpie#39

Notes for reviewers

Fork audit PR targeting audit/m-060-base, pinned to upstream b6ccd434a54bdd4761fb487d7b55aaa8bd89dbc0. It remains a draft for final human audit.

Previously posted proposals. PROBABLE-DUP comments posted before this fix retain their original command. Before copying one, check that its first argument matches the Kept tracker: link, then verify that the deduplicate confirmation lists the intended duplicate as the tracker to close.

Eval limits. Production renders the command deterministically, and pytest covers the renderer-to-selector handoff. The composition eval is a smoke test covering the documented aliases and numeric / #N arguments separated by whitespace. Comma separators, intervening words or arrows (keep, drop, into, ->, →), complete references, issue URLs and Markdown-link arguments are not normalized. A supported forward command and an unsupported reversed spelling in the same comment can pass. Plugin dispatch identity is outside these predicates' scope; the system prompt still requests security-issue-deduplicate. Quote/reference-format flags remain model-reported. These are P3, non-blocking coverage limitations.

Self-duplicate validation, uniform reference-form input handling, worded keep/drop invocation semantics, and Windows test infrastructure remain separate follow-ups.

AI-generated draft: prepared with Codex (GPT-6). AI-assisted commits carry Generated-by: attribution. Contributor review of the latest follow-up is pending.

Emit the kept tracker first in both the triage proposal and recap,
matching issue-deduplicate's existing keep/drop contract. Align the
classification and proposal-input documentation and regenerate the
marketplace runtime copies.

Round-trip both generated commands through the deduplicate CLI selector
with either number order, retain the missing-kept refusal, and add a
composition eval for the handoff direction.

Generated-by: Codex (GPT-6)
Accept the optional slash prefix and hash-marked issue numbers in the
composition eval while retaining the kept-then-dropped order check.

Explain that the model composition cases are smoke tests and that the
pytest handoff cases cover the deterministic production renderer.

Generated-by: Codex (GPT-6)
Accept the canonical command with an optional plugin namespace and
without Markdown backticks when it forms a complete command line.
Keep command, number and argument boundaries explicit.

Add a negated reversed-command assertion to the probable-duplicate case
so a response cannot pass by suggesting both merge directions. Document
the accepted forms and keep the production renderer unchanged.

Generated-by: Codex (GPT-6)
Use the selector's case-insensitive alias grammar in both command
assertions, including the marketplace issue-deduplicate invocation.
Retain the numeric argument and command-boundary checks.

Document the supported spellings and smoke-test limitations instead of
claiming coverage for repository references, URLs or Markdown links.

Generated-by: Codex (GPT-6)
Limit the documented argument coverage to whitespace-separated numbers
and hash-prefixed numbers. List comma separators, intervening words and
arrows alongside the existing reference-format exclusions.

Explain that the predicates check argument direction rather than plugin
dispatch, and that an unsupported reversed spelling can escape detection
when the response also contains a supported forward command.

Generated-by: Codex (GPT-6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant