Skip to content

chore: Add workflow to override visual regression test results - #4954

Open
jperals wants to merge 5 commits into
mainfrom
chore/required-visual-tests
Open

chore: Add workflow to override visual regression test results#4954
jperals wants to merge 5 commits into
mainfrom
chore/required-visual-tests

Conversation

@jperals

@jperals jperals commented Aug 31, 2026

Copy link
Copy Markdown
Member

Description

This is a prerequisite to making the visual regression test workflow mandatory.

Ticket: D417180663

Doc: J5RmAnz6YwPa

How has this been tested?

On this PR, two new checks are run:

On a PR without visual changes like this one, there are no functional differences.

Once this change is merged, it will be possible to test the actual override workflow (GitHub workflows can only be triggered from comments or actions if they are in main).

Review checklist

The following items are to be evaluated by the author(s) and the reviewer(s).

Correctness

  • Changes include appropriate documentation updates.
  • Changes are backward-compatible if not indicated, see CONTRIBUTING.md.
  • Changes do not include unsupported browser features, see CONTRIBUTING.md.
  • Changes were manually tested for accessibility, see accessibility guidelines.

Security

Testing

  • Changes are covered with new/existing unit tests?
  • Changes are covered with new/existing integration tests?

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.66%. Comparing base (0c1c9fd) to head (ad1342b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4954   +/-   ##
=======================================
  Coverage   97.66%   97.66%           
=======================================
  Files         959      959           
  Lines       31345    31345           
  Branches    11579    11579           
=======================================
  Hits        30614    30614           
  Misses        724      724           
  Partials        7        7           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jperals jperals reopened this Sep 4, 2026
@jperals
jperals requested a lite review from Copilot September 4, 2026 14:51
@jperals jperals changed the title chore: Make visual regression tests mandatory chore: Add workflow to override visual regression test results Sep 4, 2026
@jperals
jperals marked this pull request as ready for review September 4, 2026 14:54
@jperals
jperals requested a review from a team as a code owner September 4, 2026 14:54
@jperals
jperals requested review from cansuaa and removed request for a team September 4, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The override workflow is missing required token permissions for its Pulls API call, and the documentation currently claims PR commenting behavior that the workflow does not implement.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a per-commit override mechanism for visual regression results so maintainers can explicitly approve intentional visual diffs and allow the required “Visual regression result” check to pass without rebuilding baselines.

Changes:

  • Adds a new Override visual regression workflow that can be triggered via PR comment or manual dispatch and posts a visual-regression-override commit status.
  • Updates the reusable visual regression workflow to check for the override status and skip expensive screenshot comparison when overridden.
  • Updates the deploy workflow to surface a dedicated required “Visual regression result” gate and documents the new override flow.
File summaries
File Description
docs/RUNNING_TESTS.md Documents the override-based approval flow for intentional visual diffs.
.github/workflows/visual-regression.yml Adds a commit-status override check and skips visual shards when overridden.
.github/workflows/visual-regression-override.yml Introduces the workflow that applies the per-commit override status and reruns failed deploy jobs.
.github/workflows/deploy.yml Adds statuses read permission, passes commit SHA into the visual workflow, and adds a required “Visual regression result” gate.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/visual-regression-override.yml
Comment thread .github/workflows/deploy.yml Outdated
Comment thread docs/RUNNING_TESTS.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
jperals and others added 2 commits September 4, 2026 17:03
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants