Add advanced code review metrics view - #851
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89
|
Marking this draft while the advanced data contract is completed. BC-ALAgents currently logs BCQuality filtering/consumption but does not persist article retained, pruned, cited/used, sub-skill execution, or usage-completeness diagnostics into BC-Bench result artifacts, so this page cannot yet show the requested full performance set. |
Restore raw usage and BCQuality diagnostics to result data, aggregate them with explicit telemetry coverage, and expose them in the advanced code-review view. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89
Include immutable BC-Bench, Copilot CLI, BC-ALAgents, and BCQuality identity in summaries and aggregate grouping, and display linked pins on both code-review dashboards. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89
Track the current carry-over-focused IsHandled article name so dataset coverage remains aligned with BCQuality main. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89
Keep historical direct-agent results available while making BC PR Review the baseline and performance comparison surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89
Sun Haoran (haoranpb)
left a comment
Sun Haoran (haoranpb)
left a comment
There was a problem hiding this comment.
dayland I think lots of the changes here is not needed, if we strictly follow the versioning policy of BC-Bench.
A version of BC-Bench, e.g. v0.10.0, specifically list the version of all agent harness.
So a fixed version of BC-Bench should point to a fixed version of all agent harnesses, which should then points to fixed versions of bcquality etc
Note: the version of BC-Bench is already collected as part of the results
3 participants
Restores the PR Review diagnostics that were previously trimmed and adds a dedicated advanced dashboard for engineering analysis.
Depends on microsoft/BC-ALAgents#62 for persisted sub-results used by complete article and executed-sub-skill diagnostics.