Skip to content

feat: skip all CI checks and comment once when PR has merge conflicts - #1211

Open
rnetser wants to merge 5 commits into
mainfrom
feat/issue-1208-skip-ci-on-conflicts
Open

feat: skip all CI checks and comment once when PR has merge conflicts#1211
rnetser wants to merge 5 commits into
mainfrom
feat/issue-1208-skip-ci-on-conflicts

Conversation

@rnetser

@rnetser rnetser commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Problem

When a pushed/opened PR has merge conflicts, the webhook server still queues and runs all CI checks. Running CI against an unmergeable state wastes compute and gives the author no clear signal.

What changed

  • Guard at top of process_opened_or_synchronize_pull_request: if mergeable is False, skip ALL CI and return early.
  • Add has-conflicts label if absent and post a one-time comment (dedup via label absent->present; gated on _add_label succeeding).
  • Reset review-attribution labels (approved/commented/changes-requested/lgtm) as a normal new commit would.
  • New _pull_request_has_conflicts() helper; unknown/timeout/API-failure fail open so CI is never blocked on an indeterminate GitHub state.
  • CI resumes automatically once conflicts are resolved on the next push.

Testing

1770 passed, 90.17% coverage; ruff + mypy clean.

Review

Internal quality/guidelines/security/docs/spec reviewers all approved.

Fixes #1208

When a pushed or opened PR has merge conflicts, running CI wastes compute and gives the author no clear signal. This adds a guard at the top of process_opened_or_synchronize_pull_request: when the PR has conflicts (mergeable is False), skip all CI checks and return early.

- Add has-conflicts label if absent and post a one-time comment (gated on the label absent->present transition and on _add_label succeeding) so there is no spam on repeated conflicting pushes
- Reset review-attribution labels (approved/commented/changes-requested/lgtm) as a normal new commit would
- New _pull_request_has_conflicts() helper reuses the mergeable None-poll logic; unknown/timeout/API-failure fail open (return False) so CI is never blocked on an indeterminate GitHub state
- CI resumes automatically once conflicts are resolved on the next push

Fixes #1208

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
@myakove-bot

Copy link
Copy Markdown
Collaborator

Report bugs in Issues

Welcome! 🎉

This pull request will be automatically processed with the following features:

🔄 Automatic Actions

  • Reviewer Assignment: Reviewers are automatically assigned based on the OWNERS file in the repository root
  • Size Labeling: PR size labels (XS, S, M, L, XL, XXL) are automatically applied based on changes
  • Issue Creation: Disabled for this repository
  • Pre-commit Checks: pre-commit runs automatically if .pre-commit-config.yaml exists
  • Branch Labeling: Branch-specific labels are applied to track the target branch
  • Auto-verification: Auto-verified users have their PRs automatically marked as verified
  • Labels: All label categories are enabled (default configuration)

📋 Available Commands

PR Status Management

  • /wip - Mark PR as work in progress (adds WIP: prefix to title)
  • /wip cancel - Remove work in progress status
  • /hold - Block PR merging (PR author or approvers)
  • /hold cancel - Unblock PR merging (PR author or approvers)
  • /verified - Mark PR as verified
  • /verified cancel - Remove verification status
  • /reprocess - Trigger complete PR workflow reprocessing (useful if webhook failed or configuration changed)
  • /regenerate-welcome - Regenerate this welcome message
  • /security-override - Set security check runs to pass (maintainers only)
  • /security-override cancel - Re-run security checks

Review & Approval

  • /lgtm - Approve changes (looks good to me)
  • /approve - Approve PR (approvers only)
  • /automerge - Enable automatic merging when all requirements are met (maintainers and approvers only)
  • /assign-reviewers - Assign reviewers based on OWNERS file
  • /assign-reviewer @username - Assign specific reviewer
  • /check-can-merge - Check if PR meets merge requirements

Testing & Validation

  • /retest tox - Run Python test suite with tox
  • /retest build-container - Rebuild and test container image
  • /retest python-module-install - Test Python package installation
  • /retest pre-commit - Run pre-commit hooks and checks
  • /retest conventional-title - Validate commit message format
  • /retest all - Run all available tests

Container Operations

  • /build-and-push-container - Build and push container image (tagged with PR number)
    • Supports additional build arguments: /build-and-push-container --build-arg KEY=value

Cherry-pick Operations

  • /cherry-pick <branch> - Schedule cherry-pick to target branch when PR is merged
    • Multiple branches: /cherry-pick branch1 branch2 branch3
  • /cherry-pick-retry <branch> - Retry a failed cherry-pick (merged PRs only)

Branch Management

  • /rebase - Rebase this PR branch onto its base branch

Label Management

  • /<label-name> - Add a label to the PR
  • /<label-name> cancel - Remove a label from the PR

✅ Merge Requirements

This PR will be automatically approved when the following conditions are met:

  1. Approval: /approve from at least one approver
  2. LGTM Count: Minimum 1 /lgtm from reviewers
  3. Status Checks: All required status checks must pass
  4. No Blockers: No wip, hold, has-conflicts labels and PR must be mergeable (no conflicts)
  5. Verified: PR must be marked as verified

📊 Review Process

Approvers and Reviewers

Approvers:

  • myakove
  • rnetser

Reviewers:

  • myakove
  • rnetser
Available Labels
  • hold
  • verified
  • wip
  • lgtm
  • approve
  • automerge
AI Features
  • Conventional Title: Mode: fix (cursor/cursor-grok-4.6-high)
  • Cherry-Pick Conflict Resolution: Enabled (cursor/cursor-grok-4.6-high)
Security Checks
  • Suspicious Path Detection: Monitors paths: .claude/, .vscode/, .cursor/, .devcontainer/, .pi/, .github/workflows/, .github/actions/
  • Committer Identity Check: Verifies last committer matches PR author
  • Mandatory: Security checks block merge (use /security-override to bypass — maintainers only)

💡 Tips

  • WIP Status: Use /wip when your PR is not ready for review
  • Verification: The verified label is removed on new commits unless the push is detected as a clean rebase
  • Cherry-picking: Cherry-pick labels are processed when the PR is merged
  • Container Builds: Container images are automatically tagged with the PR number
  • Permission Levels: Some commands require approver permissions
  • Auto-verified Users: Certain users have automatic verification and merge privileges

For more information, please refer to the project documentation or contact the maintainers.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Skip CI and notify once for conflicted pull requests

✨ Enhancement 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Skip all CI work when GitHub definitively reports pull request merge conflicts.
• Add conflict labels and a deduplicated resolution comment while clearing stale review attribution.
• Fail open on unknown mergeability and resume CI after conflicts are resolved.
Diagram

graph TD
  A["PR Webhook"] --> B["Mergeability Probe"] --> C{"Confirmed conflicts?"}
  C -- "Yes" --> D["Conflict Notice"] --> E["Reset Review Labels"] --> F["Skip CI"]
  C -- "No or unknown" --> G["Normal CI Workflow"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Shared tri-state mergeability resolver
  • ➕ Avoids duplicating polling logic already used by merge-state labeling.
  • ➕ Preserves explicit conflict, mergeable, and unknown states for each caller.
  • ➕ Centralizes timeout, cancellation, and GitHub API failure behavior.
  • ➖ Requires a broader refactor of established merge-state labeling.
  • ➖ Expands regression scope beyond the new CI guard.
2. Comment-based deduplication
  • ➕ Guarantees one notification even if users manually remove the conflict label.
  • ➕ Decouples user notification history from label state.
  • ➖ Requires fetching and scanning existing comments.
  • ➖ Adds API cost and needs a stable hidden comment marker.

Recommendation: The localized early guard is appropriate for a focused, low-risk delivery and label-transition deduplication avoids additional comment API calls. A shared tri-state mergeability resolver would be the stronger long-term design because similar polling already exists in merge-state labeling, but that consolidation is better handled separately to avoid widening this PR.

Files changed (2) +247 / -17

Enhancement (1) +85 / -1
pull_request_handler.pyGate PR CI execution on confirmed merge conflicts +85/-1

Gate PR CI execution on confirmed merge conflicts

• Adds fail-open mergeability detection with bounded polling and cancellation-safe error handling. Confirmed conflicts add the conflict label, post one guidance comment after a successful label transition, clear stale review labels, and return before any CI or reviewer tasks are queued.

webhook_server/libs/handlers/pull_request_handler.py

Tests (1) +162 / -16
test_pull_request_handler.pyCover conflict gating, notification deduplication, and fail-open behavior +162/-16

Cover conflict gating, notification deduplication, and fail-open behavior

• Adds tests for newly detected and previously labeled conflicts, label-add failures, normal CI execution, GitHub API failures, definitive polling results, and polling timeouts. Assertions verify that conflicted PRs bypass checks and reviewer assignment while non-conflicted or indeterminate PRs continue through CI.

webhook_server/tests/test_pull_request_handler.py

@qodo-code-review

qodo-code-review Bot commented Sep 2, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Cleanup failure resumes Oracle 🐞 Bug ☼ Reliability ⭐ New
Description
If remove_labels_when_pull_request_sync() fails while processing a conflicted PR,
_handle_conflicted_pull_request() propagates the exception instead of preserving the successful
conflict decision. The caller then interprets the exception as skipped_due_to_conflicts=False and
schedules Test Oracle, violating the all-CI skip guarantee.
Code

webhook_server/libs/handlers/pull_request_handler.py[1242]

+        await self.remove_labels_when_pull_request_sync(pull_request=pull_request)
Relevance

●●● Strong

Recent precedents accept best-effort failure handling to preserve successful workflow outcomes after
noncritical API failures.

PR-#1125
PR-#1109
PR-#968

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The newly added unconditional cleanup await can raise when its GitHub label-list call exhausts
retries. In both opened and synchronized event handlers, gathered exceptions are not equal to
True, so skipped_due_to_conflicts becomes false and call_test_oracle() is scheduled.

webhook_server/libs/handlers/pull_request_handler.py[1238-1248]
webhook_server/libs/handlers/pull_request_handler.py[1532-1556]
webhook_server/libs/handlers/pull_request_handler.py[273-284]
webhook_server/libs/handlers/pull_request_handler.py[319-328]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failure while removing stale review labels propagates from the conflicted-PR path. The webhook caller consequently loses the `True` skip result and may schedule Test Oracle even though conflicts were definitively detected.

## Issue Context
Notification failures are already best-effort, but the subsequent cleanup operation is not. Preserve cancellation while treating ordinary cleanup failures as best-effort so the method still reaches the conflict completion path and returns the skip result.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1238-1248]
- webhook_server/libs/handlers/pull_request_handler.py[1357-1366]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Failed comment never retries ✓ Resolved 🐞 Bug ☼ Reliability
Description
The code commits the has-conflicts label before creating the comment, so a permanent comment API
failure exits with the deduplication marker already present. Every later conflicted webhook then
sees the label and permanently skips the missing notification.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1184-1187]

+                if label_added:
+                    await github_api_call(
+                        pull_request.create_issue_comment,
+                        "⚠️ CI checks were skipped because this pull request has merge conflicts.\n\n"
Relevance

●●● Strong

PR #1125 specifically established best-effort comment posting after workflow state is established.

PR-#1125

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Label addition completes before the comment call, exhausted GitHub retries propagate an exception,
and subsequent executions bypass comment creation whenever the label is present.

webhook_server/libs/handlers/pull_request_handler.py[1176-1198]
webhook_server/libs/handlers/labels_handler.py[179-181]
webhook_server/utils/github_retry.py[101-128]
PR-#1125

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed comment request leaves the conflict label installed, causing all future webhook deliveries to treat the missing comment as already posted.

## Issue Context
Use a dedicated bot-authored comment marker for deduplication, or roll back/retry the marker when comment creation fails while still completing cleanup.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1176-1198]
- webhook_server/utils/github_retry.py[101-128]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Test Oracle still runs ✓ Resolved 🐞 Bug ≡ Correctness
Description
After the conflict branch returns, the opened and synchronize callers still unconditionally schedule
call_test_oracle, which can perform a 300-second external analysis when those triggers are
configured. Conflicted PRs therefore still consume CI-related compute despite the claimed all-CI
skip.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1207-1209]

+            if self.ctx:
+                self.ctx.complete_step("pr_workflow_setup", skipped_due_to_conflicts=True)
+            return
Relevance

●●● Strong

Accepted precedent requires preventing unrelated background Test Oracle work during specialized
workflow paths.

PR-#1002
PR-#1060

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Both event paths schedule Test Oracle outside and after the changed method, while Test Oracle
performs a potentially long external analysis when its trigger is enabled.

webhook_server/libs/handlers/pull_request_handler.py[270-289]
webhook_server/libs/handlers/pull_request_handler.py[295-329]
webhook_server/libs/test_oracle.py[33-43]
webhook_server/libs/test_oracle.py[88-100]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The conflict result only returns from the inner workflow, so callers still schedule Test Oracle analysis for configured opened or synchronize triggers.

## Issue Context
Expose the conflict-skip outcome to the caller and avoid scheduling Test Oracle when the PR is definitively conflicted.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[270-289]
- webhook_server/libs/handlers/pull_request_handler.py[295-329]
- webhook_server/libs/handlers/pull_request_handler.py[1168-1209]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View high (1)
4. Disabled label suppresses comment ✓ Resolved 🐞 Bug ≡ Correctness
Description
The conflict comment is posted only when _add_label returns true, but that method returns false
whenever the configurable has-conflicts category is disabled. Such repositories silently skip CI
without giving the author the promised explanatory comment.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1181-1185]

+                label_added = await self.labels_handler._add_label(
+                    pull_request=pull_request, label=HAS_CONFLICTS_LABEL_STR
+                )
+                if label_added:
+                    await github_api_call(
Relevance

●● Moderate

The behavior conflicts with the stated notification intent, but no close precedent covers disabled
label categories.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
has-conflicts is a configurable label category, and _add_label explicitly returns false for
disabled categories; the changed code only comments on a true return.

webhook_server/libs/handlers/pull_request_handler.py[1179-1192]
webhook_server/libs/handlers/labels_handler.py[86-94]
webhook_server/libs/handlers/labels_handler.py[140-158]
webhook_server/utils/constants.py[79-89]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Repositories that disable the `has-conflicts` label category receive no conflict notification because comment creation is gated on successful label addition.

## Issue Context
Keep comment deduplication functional even when conflict labels are disabled, such as by checking for a bot-authored marker comment.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1176-1192]
- webhook_server/libs/handlers/labels_handler.py[86-94]
- webhook_server/libs/handlers/labels_handler.py[140-158]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

5. Cancelled polling continues requests ✓ Resolved 🐞 Bug ☼ Reliability
Description
Cancelling _get_definitive_mergeable only cancels the await on the sampler thread; the nested
asyncio.run(github_api_call(...)) has no cancellation signal and can continue retries and issue
GitHub requests after the webhook task exits. This leaves background work running during
cancellation or shutdown and can consume threads or make late API calls.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1149-1152]

+                return asyncio.run(
+                    github_api_call(
+                        lambda: self.github_webhook.repository.get_pull(pr_number).mergeable,
+                        logger=self.logger,
Relevance

●●● Strong

Recent precedents prioritize propagating cancellation and managing background work in pull-request
handlers.

PR-#1109
PR-#968
PR-#1002

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The polling callback creates an independent event loop around github_api_call, while the sampler
itself runs under asyncio.to_thread; the outer cancellation handler merely re-raises and never
signals either worker. github_api_call may keep executing threaded calls and exponential retry
sleeps until that independent loop completes.

webhook_server/libs/handlers/pull_request_handler.py[1146-1175]
webhook_server/utils/github_retry.py[101-125]
PR-#1109

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The mergeability sampler runs a nested `asyncio.run(github_api_call(...))` in a worker thread. Cancelling the parent coroutine cannot stop that nested retry loop, allowing GitHub requests and retry waits to continue after the webhook task has exited.

## Issue Context
`github_api_call` can perform multiple threaded attempts with asynchronous backoff. The outer coroutine re-raises cancellation, but no cancellation signal reaches `_run_sampler` or the nested event loop.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1145-1160]
- webhook_server/libs/handlers/pull_request_handler.py[1163-1175]
- webhook_server/utils/github_retry.py[101-125]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Polling bypasses github_api_call() ✓ Resolved 📘 Rule violation ☼ Reliability
Description
The polling callback directly invokes repository.get_pull() and accesses .mergeable from a live
PyGithub object, with only the enclosing sampler sent through asyncio.to_thread(). This bypasses
the required GitHub API wrapper and its standardized retry behavior.
Code

webhook_server/libs/handlers/pull_request_handler.py[1160]

+                func=lambda: self.github_webhook.repository.get_pull(pr_number).mergeable,
Relevance

●●● Strong

Recent precedents accept wrapping PyGithub property access and pagination in github_api_call or
worker threads.

PR-#1102
PR-#1060
PR-#894

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 1523063 requires every PyGithub method call and property access to occur inside
github_api_call(). The new sampler callback directly evaluates
self.github_webhook.repository.get_pull(pr_number).mergeable, while github_api_call() is the
repository's designated threaded retry wrapper.

Rule 1523063: Wrap all PyGithub API interactions in github_api_call()
webhook_server/libs/handlers/pull_request_handler.py[1157-1162]
webhook_server/utils/github_retry.py[65-76]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The mergeability polling callback directly uses the PyGithub repository instead of routing the interaction through `github_api_call()`.

## Issue Context
Preserve the bounded polling behavior while ensuring every `get_pull()` call and `.mergeable` access executes through `github_api_call()`. Avoid wrapping the entire polling window as one retryable operation, which could restart the timeout budget.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1147-1174]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Conflict marker spoofing ✓ Resolved 🐞 Bug ⛨ Security
Description
The deduplication predicate trusts any comment containing CONFLICT_COMMENT_MARKER without
verifying its author. Any user who can comment can insert the marker and prevent the bot from
posting the conflict notification while CI is still skipped.
Code

webhook_server/libs/handlers/pull_request_handler.py[1217]

+                if not any(body is not None and CONFLICT_COMMENT_MARKER in body for body in existing_comment_bodies):
Relevance

●●● Strong

PR #1109 accepted this exact spoofing risk and required trusted bot-author verification for comment
deduplication.

PR-#1109

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The code reduces comments to bodies and suppresses notification solely on marker presence,
discarding author identity. Existing deduplication elsewhere checks both body content and a trusted
commenter login, and past review history documents this same spoofing pattern.

webhook_server/libs/handlers/pull_request_handler.py[1202-1217]
webhook_server/libs/handlers/pull_request_handler.py[1432-1435]
PR-#1109

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Conflict-comment deduplication trusts the hidden marker regardless of who authored the comment. Require the marker to appear in a comment from a trusted webhook/bot identity so regular users cannot suppress the authoritative notification.

## Issue Context
Use a stable configured set of trusted bot identities rather than only the currently selected API user, since credentials may rotate between deliveries.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1207-1217]
- webhook_server/tests/test_pull_request_handler.py[783-807]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View medium (5)
8. Unwrapped pull_request.number access ✓ Resolved 📘 Rule violation ☼ Reliability
Description
The new conflict polling path reads number directly from a live PyGithub PullRequest. All
PyGithub property access must occur through github_api_call().
Code

webhook_server/libs/handlers/pull_request_handler.py[1139]

+            pr_number = pull_request.number
Relevance

●●● Strong

Recent precedents require wrapping live PyGithub property access in github_api_call or equivalent.

PR-#1102
PR-#1060

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 1523063 requires every property access on a PyGithub object to occur within
github_api_call(). The new helper wraps pull_request.mergeable but then directly evaluates
pull_request.number.

Rule 1523063: Wrap all PyGithub API interactions in github_api_call()
webhook_server/libs/handlers/pull_request_handler.py[1131-1140]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The conflict polling path directly accesses `pull_request.number`, contrary to the requirement that PyGithub property access use `github_api_call()`.

## Issue Context
`pull_request` is annotated as a PyGithub `PullRequest`; the adjacent `mergeable` access is already correctly wrapped.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1131-1146]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


9. Bookkeeping failure skips cleanup ✓ Resolved 🐞 Bug ☼ Reliability
Description
Failures while reading or adding labels or posting the comment escape before
remove_labels_when_pull_request_sync, leaving stale approval and review-attribution labels on the
conflicted commit. The event callers gather exceptions and continue, so this required reset is not
retried within the delivery.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1174-1176]

+        has_conflicts = await self._pull_request_has_conflicts(pull_request=pull_request)
+        if has_conflicts:
+            current_labels = await self.labels_handler.pull_request_labels_names(pull_request=pull_request)
Relevance

●●● Strong

Team consistently requests exception handling so bookkeeping failures cannot leave workflow state
stale.

PR-#1109
PR-#968

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The changed side effects precede cleanup without local exception handling; callers consume task
exceptions with return_exceptions=True, while label-list failures propagate from
github_api_call.

webhook_server/libs/handlers/pull_request_handler.py[1174-1209]
webhook_server/libs/handlers/labels_handler.py[111-120]
webhook_server/libs/handlers/pull_request_handler.py[261-275]
webhook_server/libs/handlers/pull_request_handler.py[295-319]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Optional conflict-label and comment operations can throw before the required review-attribution label reset and workflow-step completion.

## Issue Context
Make notification bookkeeping best-effort and place mandatory cleanup and context completion in a cancellation-safe `finally` path.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1174-1209]
- webhook_server/libs/handlers/pull_request_handler.py[261-275]
- webhook_server/libs/handlers/pull_request_handler.py[295-319]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


10. Concurrent webhooks duplicate comments 🐞 Bug ☼ Reliability
Description
The read/add/comment sequence is not serialized, and _add_label repeats the same non-atomic
existence check before adding. Concurrent deliveries can both observe no label, both return
successful after the label exists, and both post the supposedly one-time comment.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1176-1181]

+            current_labels = await self.labels_handler.pull_request_labels_names(pull_request=pull_request)
+            conflicts_label_present = HAS_CONFLICTS_LABEL_STR in current_labels
+
+            label_added = False
+            if not conflicts_label_present:
+                label_added = await self.labels_handler._add_label(
Relevance

●●● Strong

Recent accepted findings address duplicate-comment races and require robust comment deduplication.

PR-#1109

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
There is no lock around the changed read/add/comment sequence; _add_label also separates its
existence lookup from the eventual add and returns based on observing the label afterward.

webhook_server/libs/handlers/pull_request_handler.py[1176-1192]
webhook_server/libs/handlers/labels_handler.py[156-181]
webhook_server/libs/handlers/labels_handler.py[183-194]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Concurrent conflict webhooks can pass both label-existence checks and each create the conflict comment.

## Issue Context
Serialize conflict notification per repository/PR or deduplicate against a bot-authored comment marker after label creation.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1176-1192]
- webhook_server/libs/handlers/labels_handler.py[156-181]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


11. Conflict tests spawn threads ✓ Resolved 📘 Rule violation ▣ Testability
Description
The new mergeability tests exercise the real github_api_call(), which delegates to
asyncio.to_thread, while mocking GitHub properties and methods without patching
asyncio.to_thread. This permits actual worker-thread execution and violates the required test
isolation pattern.
Code

webhook_server/tests/test_pull_request_handler.py[887]

+            result = await pull_request_handler._pull_request_has_conflicts(pull_request=mock_pull_request)
Relevance

●●● Strong

Recent testing precedents explicitly require patching asyncio.to_thread when exercising
github_api_call with mocked GitHub objects.

PR-#1156
PR-#1172

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Rule 1523210 requires tests that mock GitHub interactions exercised through github_api_call() to
patch asyncio.to_thread. The polling and timeout tests mock mergeability and get_pull behavior
and invoke the real helper without such a patch, while github_api_call() executes its callable
using asyncio.to_thread.

Rule 1523210: Patch asyncio.to_thread in tests mocking github_api_call
webhook_server/tests/test_pull_request_handler.py[875-907]
webhook_server/utils/github_retry.py[101-104]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New tests exercise `_pull_request_has_conflicts()` with mocked GitHub objects but leave `asyncio.to_thread` unpatched, so the real `github_api_call()` creates worker threads.

## Issue Context
Patch `asyncio.to_thread` with an awaitable inline implementation or controllable `AsyncMock` in each affected test that uses the real conflict helper.

## Fix Focus Areas
- webhook_server/tests/test_pull_request_handler.py[825-848]
- webhook_server/tests/test_pull_request_handler.py[875-907]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


12. Polling exceeds timeout budget ✓ Resolved 🐞 Bug ➹ Performance
Description
The entire 30-second TimeoutSampler is wrapped as one retryable github_api_call, so a late
transient API failure restarts the complete polling window. Up to five full polling attempts plus
exponential backoff can greatly exceed the documented 30-second limit and tie up webhook processing
workers.
Code

webhook_server/libs/handlers/pull_request_handler.py[R1143-1146]

+                for sample in TimeoutSampler(
+                    wait_timeout=30,
+                    sleep=5,
+                    func=lambda: repository.get_pull(pr_number).mergeable,
Relevance

●● Moderate

No close timeout-sampler retry precedent found; concern is plausible but lacks decisive team
evidence.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The sampler owns a 30-second timeout, while github_api_call retries the supplied callable up to
four times; because the supplied callable is the whole sampler, every retry starts a new timeout
window.

webhook_server/libs/handlers/pull_request_handler.py[1138-1157]
webhook_server/utils/github_retry.py[72-76]
webhook_server/utils/github_retry.py[101-125]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Retrying the complete synchronous sampler restarts its 30-second timeout and allows mergeability detection to substantially exceed its stated deadline.

## Issue Context
Use one outer deadline and wrap each individual GitHub refresh in retry handling without resetting the overall polling budget.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1138-1157]
- webhook_server/utils/github_retry.py[101-125]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 13 rules
Review mode: ⚖️ Balanced

Grey Divider

Tip of the day
💡 Did you know, you can turn on the rule miner and Qodo learns your standards from review history

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Comment thread webhook_server/tests/test_pull_request_handler.py
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Comment thread webhook_server/libs/handlers/pull_request_handler.py
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
Extract shared _get_definitive_mergeable() used by both the conflict guard and label_pull_request_by_merge_state (removes duplicated mergeable None-poll).

- Wrap pull_request.number in github_api_call (rule violation)
- Bound the mergeable poll to a single 30s deadline; retry each refresh individually instead of the whole window
- Skip test-oracle scheduling for conflicted PRs (process_opened_or_synchronize_pull_request now returns a conflict-skip bool consumed by callers)
- Dedup the conflict comment via a hidden marker comment instead of the has-conflicts label, so a disabled label category or a prior failed comment no longer suppresses or blocks retry of the notification
- Run mandatory review-label reset + ctx bookkeeping best-effort in all cases even if the optional label/comment notification raises
- Patch asyncio.to_thread in conflict tests to avoid spawning real worker threads

Refs #1208

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
@rnetser

rnetser commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

/qodo

The following review comments were reviewed and a decision was made:

webhook_server/libs/handlers/pull_request_handler.py:1207 (qodo bug) — Test Oracle still runs

Addressed: Addressed in 7e50890: process_opened_or_synchronize_pull_request now returns a conflict-skip bool; both opened and synchronize callers skip scheduling call_test_oracle when the PR is conflicted.

webhook_server/libs/handlers/pull_request_handler.py:1184 (qodo bug) — Failed comment never retries

Addressed: Addressed in 7e50890: dedup now uses a hidden marker comment rather than the label. If a prior comment attempt failed, the marker is absent so the next delivery retries the post.

webhook_server/libs/handlers/pull_request_handler.py:1181 (qodo bug) — Disabled label suppresses comment

Addressed: Addressed in 7e50890: the conflict comment is no longer gated on _add_label success. It is deduped via a hidden marker comment, so a disabled has-conflicts label category no longer suppresses the notification.

webhook_server/libs/handlers/pull_request_handler.py:1139 (qodo rule violation) — Unwrapped pull_request.number access

Addressed: Addressed in 7e50890: pull_request.number is now accessed via github_api_call() inside the new shared _get_definitive_mergeable() helper.

webhook_server/tests/test_pull_request_handler.py:887 (qodo rule violation) — Conflict tests spawn threads

Addressed: Addressed in 7e50890: added an autouse fixture patching asyncio.to_thread with an inline AsyncMock so github_api_call runs inline without spawning real worker threads.

webhook_server/libs/handlers/pull_request_handler.py:1176 (qodo bug) — Concurrent webhooks duplicate comments

Addressed: Addressed in 7e50890: dedup now checks existing comment bodies for a hidden marker before posting. Full cross-delivery serialization remains best-effort (documented) and out of scope.

webhook_server/libs/handlers/pull_request_handler.py:1174 (qodo bug) — Bookkeeping failure skips cleanup

Addressed: Addressed in 7e50890: optional label/comment notification is wrapped in best-effort try/except (logs, re-raises CancelledError); mandatory remove_labels_when_pull_request_sync + ctx.complete_step + return always run.

webhook_server/libs/handlers/pull_request_handler.py:1143 (qodo bug) — Polling exceeds timeout budget

Addressed: Addressed in 7e50890: the mergeable poll now uses a single bounded 30s deadline in async; each individual refresh is wrapped by github_api_call rather than wrapping the whole 30s window, so retries no longer multiply the budget.


Assisted-by: PI (claude-opus-4-8)

@myakove-bot myakove-bot removed the size/L label Sep 2, 2026
@qodo-code-review

Copy link
Copy Markdown

The following review comments were reviewed and a decision was made:

webhook_server/libs/handlers/pull_request_handler.py:1207 (qodo bug) — Test Oracle still runs

Addressed: Addressed in 7e50890: process_opened_or_synchronize_pull_request now returns a conflict-skip bool; both opened and synchronize callers skip scheduling call_test_oracle when the PR is conflicted.

webhook_server/libs/handlers/pull_request_handler.py:1184 (qodo bug) — Failed comment never retries

Addressed: Addressed in 7e50890: dedup now uses a hidden marker comment rather than the label. If a prior comment attempt failed, the marker is absent so the next delivery retries the post.

webhook_server/libs/handlers/pull_request_handler.py:1181 (qodo bug) — Disabled label suppresses comment

Addressed: Addressed in 7e50890: the conflict comment is no longer gated on _add_label success. It is deduped via a hidden marker comment, so a disabled has-conflicts label category no longer suppresses the notification.

webhook_server/libs/handlers/pull_request_handler.py:1139 (qodo rule violation) — Unwrapped pull_request.number access

Addressed: Addressed in 7e50890: pull_request.number is now accessed via github_api_call() inside the new shared _get_definitive_mergeable() helper.

webhook_server/tests/test_pull_request_handler.py:887 (qodo rule violation) — Conflict tests spawn threads

Addressed: Addressed in 7e50890: added an autouse fixture patching asyncio.to_thread with an inline AsyncMock so github_api_call runs inline without spawning real worker threads.

webhook_server/libs/handlers/pull_request_handler.py:1176 (qodo bug) — Concurrent webhooks duplicate comments

Addressed: Addressed in 7e50890: dedup now checks existing comment bodies for a hidden marker before posting. Full cross-delivery serialization remains best-effort (documented) and out of scope.

webhook_server/libs/handlers/pull_request_handler.py:1174 (qodo bug) — Bookkeeping failure skips cleanup

Addressed: Addressed in 7e50890: optional label/comment notification is wrapped in best-effort try/except (logs, re-raises CancelledError); mandatory remove_labels_when_pull_request_sync + ctx.complete_step + return always run.

webhook_server/libs/handlers/pull_request_handler.py:1143 (qodo bug) — Polling exceeds timeout budget

Addressed: Addressed in 7e50890: the mergeable poll now uses a single bounded 30s deadline in async; each individual refresh is wrapped by github_api_call rather than wrapping the whole 30s window, so retries no longer multiply the budget.


Assisted-by: PI (claude-opus-4-8)

The update addresses findings 1–5, 7, and 8 as described. However, finding 6 remains active: checking for CONFLICT_COMMENT_MARKER before posting is not atomic, so concurrent webhook deliveries can both observe no marker and create duplicate comments. The code explicitly documents this as “best-effort,” so this finding is not fully resolved unless duplicate comments are acceptable by design.

Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 7e50890

Address follow-up Qodo review findings on PR #1211.

- Reuse TimeoutSampler (single bounded 30s window, sleep 5) run once via asyncio.to_thread with an
 exceptions_dict for transient GitHub 5xx/connection errors, instead of a hand-rolled deadline loop; keeps one bounded
 budget without github_api_call retry multiplication
   - Validate the conflict-comment dedup marker against trusted bot identities (security_trusted_committers +
 app_bot_login); a marker in an untrusted user comment no longer suppresses the authoritative notification (prevents
 marker spoofing). Empty trusted set validates no markers (fail-safe, never trust all authors)

Refs #1208

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 5cf6638

Address Qodo finding that the TimeoutSampler poll callback accessed repository.get_pull().mergeable directly, bypassing the standardized retry wrapper.

- Keep TimeoutSampler for the single bounded 30s window (no whole-window retry multiplication)
   - Route each individual mergeable refresh through github_api_call via an asyncio.run bridge inside the sampler's worker-thread callback, so every get_pull()/.mergeable access uses standardized retry
   - Drop the redundant sampler exceptions_dict and now-unused imports (github_api_call is the single retry authority)

Refs #1208

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
Comment thread webhook_server/libs/handlers/pull_request_handler.py Outdated
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 646a7e2

…n-safe

Address Qodo finding that the nested asyncio.run(github_api_call) poll bridge could not be cancelled and kept issuing GitHub requests after task exit.

- Replace TimeoutSampler + asyncio.run bridge in _get_definitive_mergeable with a native async deadline loop: per-refresh github_api_call (standardized retry), single bounded 30s budget, cancellation propagates through await
   - Split the 178-line process_opened_or_synchronize_pull_request into _handle_conflicted_pull_request, _queue_pull_request_setup_tasks, and _run_pull_request_ci_tasks; orchestrator is now a short dispatcher (behavior unchanged)
   - Drop the unneeded [[tool.mypy.overrides]] for timeout_sampler (import removed; pyproject matches main)

Refs #1208

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: rnetser <rnetser@redhat.com>
# (approved/commented/changes-requested/lgtm) as a normal new commit would. On non-clean
# synchronize this may run alongside the caller's reset; duplicate removals are benign and
# idempotent.
await self.remove_labels_when_pull_request_sync(pull_request=pull_request)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Cleanup failure resumes oracle 🐞 Bug ☼ Reliability

If remove_labels_when_pull_request_sync() fails while processing a conflicted PR,
_handle_conflicted_pull_request() propagates the exception instead of preserving the successful
conflict decision. The caller then interprets the exception as skipped_due_to_conflicts=False and
schedules Test Oracle, violating the all-CI skip guarantee.
Agent Prompt
## Issue description
A failure while removing stale review labels propagates from the conflicted-PR path. The webhook caller consequently loses the `True` skip result and may schedule Test Oracle even though conflicts were definitively detected.

## Issue Context
Notification failures are already best-effort, but the subsequent cleanup operation is not. Preserve cancellation while treating ordinary cleanup failures as best-effort so the method still reaches the conflict completion path and returns the skip result.

## Fix Focus Areas
- webhook_server/libs/handlers/pull_request_handler.py[1238-1248]
- webhook_server/libs/handlers/pull_request_handler.py[1357-1366]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 5fedacc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: skip all CI checks and comment once when PR has merge conflicts

2 participants