task: add GPU measurement-validity and profiler-attribution guidance to inference-benchmark-design #455

Activity

1. amy-why-3459 commented on Nov 15, 2026

   

   amy-why-3459

   on Nov 15, 2026 – with Cursor

   Collaborator

   More actions

   Thanks, the scope is well bounded: documentation only, status stays candidate / empirical not_run, and it is complementary to #454 without a hard dependency. The associated draft is #456 at 8c0b894a2dc9b8572700ca3f80b119e75f2377db. A few points before acceptance:

   1. Owner and acceptance. The issue is still Proposed, and the goal-acceptance owner is TBD. Please have that owner recorded before treating the slice as accepted.
   2. Matrix placement. The new docs/reference-matrix.md row in knowledge: add GPU measurement-validity and profiler attribution guidance #456 sits under "Task-specific benchmark selection (issue [RFC] Benchmark selection by task for InferRSI #60)", so the generated docs/skill-coverage.md also lists it as an [RFC] Benchmark selection by task for InferRSI #60 source, and the searchspace note was inserted under the same heading. This guidance is general GPU measurement methodology, not a task-selection source. Please move the matrix row and the searchspace paragraph to a dedicated short section (similar to the later "Faster CosyVoice evaluation source context" block) so the generated coverage label is accurate.
   3. Adoption wording. "Require an explanation before adoption" should remain clearly candidate guidance, not a repository-wide gate, until an owner decision or rfc: 可解释的改进——用 profile 证据和配对统计判定 PASS #454 says otherwise. The SKILL.md pointer already calls it candidate guidance; please keep that if the paragraph is edited.
   4. Author-reported numbers. The four task-supplied figures are clearly labelled unverified and not reusable thresholds; please keep them that way.
   5. One source I checked. Triton v3.2.0 is 9641643da6c52000c807b5eeed05edaec4402a67, and do_bench does a write-based cache.zero_() flush, includes that flush in the preliminary estimate, and places per-sample events only around fn. That matches the reference. I did not re-download the NVIDIA/PyTorch documents to re-check the retained hashes.

   #456 is also based on a92c77ccaed68426c83154cff7619745f34d0058; current main is cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332 (#453), which added C1 assertions to tests/check_installed.py while leaving the resource count at 923. Please rebase before merge and regenerate records if the source-revision commit changes. I'll comment on the PR separately for the matrix section and the rebase.
2. Dong1017 commented on Nov 15, 2026

   

   Dong1017

   on Nov 15, 2026

   Collaborator

   More actions

   @BeatSeat — steward decision for @Dong1017 (authorized Dong-bot / InferRSI on-call delegate; comment posted as Dong1017).

   Decision: accept scope (with conditions) for this documentation-only knowledge slice.

   Field	Record
   Status	Proposed → Accepted (knowledge contribution scope)
   Accountable parent / goal-acceptance owner	Dong1017
   Slice implementer	BeatSeat
   Associated draft	#456 @ 8c0b894a2dc9b8572700ca3f80b119e75f2377db

   Accepted

   • Owner-local reference under methods/inference-benchmark-design plus a short SKILL.md pointer.
   • Candidate / empirical not_run / efficacy not_evaluated preserved; no new skill, runner, numeric gate, GPU execution, or performance claim.
   • Complementary to rfc: 可解释的改进——用 profile 证据和配对统计判定 PASS #454 / harness: explain gains with profile evidence and paired statistics #457; no hard dependency.

   Conditions before ready-for-review / Approve (also recorded on #456)

   1. Move the reference-matrix row and searchspace note out of the issue [RFC] Benchmark selection by task for InferRSI #60 section into a dedicated short section, then regenerate coverage/records.
   2. Rebase onto current main (cd8fcb45 / Package shared technical contracts and retire root modules #453) and refresh declaration head_sha / projections if the source-revision commit changes.
   3. Keep “require an explanation before adoption” clearly candidate guidance, not a repository-wide gate, until an owner decision or rfc: 可解释的改进——用 profile 证据和配对统计判定 PASS #454 says otherwise.

   This acceptance authorizes the scoped knowledge work. It is not Approve of #456 and not Issue closure.