Skip to content

fix(listener): messages semaphore release - #2763

Open
pindo696 wants to merge 1 commit into
RedHatInsights:masterfrom
pindo696:semaphore-release-listener
Open

pindo696 wants to merge 1 commit into
RedHatInsights:masterfrom
pindo696:semaphore-release-listener

Conversation

@pindo696

@pindo696 pindo696 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Messages semaphore wrapped into handle single consume message so we have better control over semaphore acquires and releases instead of relying on the component that does not know about the semaphore resultion into a single semaphore acquire and release. RHINENG-33634

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Fix message consumption so semaphore capacity is reliably restored after every message.

Bug Fixes:

  • Ensure the message-processing semaphore is released exactly once for both successful and failed message handling.

Enhancements:

  • Centralize semaphore ownership at the per-message handler while keeping inventory and advisor consumers focused on message processing.
  • Await message processing within each background task to maintain correct processing and tracing behavior.

Tests:

  • Add coverage verifying semaphore release after successful processing and malformed-message failures.

@sourcery-ai

sourcery-ai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Implementation

Move semaphore release into a per-message task wrapper and await message processing within that task, centralizing semaphore ownership at the task boundary.

  • listener/listener.py L249-258, L271-274 (+4 small edits)

Tests

Add tests checking that the per-message wrapper releases the semaphore after invalid-message and successful processing paths.

  • tests/listener_tests/test_listener_semaphore.py L1-97
  • tests/listener_tests/test_listener_tracing.py L83 (removed lines), L142 (removed lines)

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/listener_tests/test_listener_semaphore.py" line_range="67" />
<code_context>
+    record = create_consumer_record(invalid_json)
+
+    # Use the exact method name from listener_2.py
+    await listener._handle_single_consume_message(record)
+
+    # Assert it was released (if we try to release a fully bounded semaphore, it raises ValueError)
</code_context>
<issue_to_address>
**Processing errors go untested**

When message processing raises an exception after JSON decoding, `invalid_json` raises `JSONDecodeError`, which `_consume_message` catches and returns from, so `_handle_single_consume_message` completes normally and the semaphore assertion passes even if cleanup after a processing exception is broken; regressions in release-on-exception go unnoticed.

Use valid JSON and make the inventory or advisor processor raise an exception, then assert that the semaphore was released.
</issue_to_address>

### Comment 2
<location path="tests/listener_tests/test_listener_semaphore.py" line_range="94" />
<code_context>
+    await listener._handle_single_consume_message(record)
+
+    # Assert it was released (if we try to release a fully bounded semaphore, it raises ValueError)
+    with pytest.raises(ValueError):
+        listener.max_messages_semaphore.release()
+
</code_context>
<issue_to_address>
**Early release still passes**

When the handler releases the semaphore slot before message processing completes, `test_semaphore_released_on_success` (lines 91-97) checks the semaphore only after the handler finishes; releasing the slot before awaiting processing still leaves it full at line 94 and lets the processor assertion pass, so the test accepts premature release.

Block the mocked processor on an event, assert the slot remains acquired while processing is blocked, then unblock it and verify the slot is released.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

record = create_consumer_record(invalid_json)

# Use the exact method name from listener_2.py
await listener._handle_single_consume_message(record)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Processing errors go untested

When message processing raises an exception after JSON decoding, invalid_json raises JSONDecodeError, which _consume_message catches and returns from, so _handle_single_consume_message completes normally and the semaphore assertion passes even if cleanup after a processing exception is broken; regressions in release-on-exception go unnoticed.

Use valid JSON and make the inventory or advisor processor raise an exception, then assert that the semaphore was released.

Prompt for AI agents
In `tests/listener_tests/test_listener_semaphore.py` at line 67:

**Processing errors go untested**

When message processing raises an exception after JSON decoding, `invalid_json` raises `JSONDecodeError`, which `_consume_message` catches and returns from, so `_handle_single_consume_message` completes normally and the semaphore assertion passes even if cleanup after a processing exception is broken; regressions in release-on-exception go unnoticed.

Use valid JSON and make the inventory or advisor processor raise an exception, then assert that the semaphore was released.

await listener._handle_single_consume_message(record)

# Assert it was released back to its maximum bound
with pytest.raises(ValueError):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Early release still passes

When the handler releases the semaphore slot before message processing completes, test_semaphore_released_on_success (lines 91-97) checks the semaphore only after the handler finishes; releasing the slot before awaiting processing still leaves it full at line 94 and lets the processor assertion pass, so the test accepts premature release.

Block the mocked processor on an event, assert the slot remains acquired while processing is blocked, then unblock it and verify the slot is released.

Prompt for AI agents
In `tests/listener_tests/test_listener_semaphore.py` at line 94:

**Early release still passes**

When the handler releases the semaphore slot before message processing completes, `test_semaphore_released_on_success` (lines 91-97) checks the semaphore only after the handler finishes; releasing the slot before awaiting processing still leaves it full at line 94 and lets the processor assertion pass, so the test accepts premature release.

Block the mocked processor on an event, assert the slot remains acquired while processing is blocked, then unblock it and verify the slot is released.

@codecov-commenter

codecov-commenter commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.84%. Comparing base (211912d) to head (efbffd0).
⚠️ Report is 91 commits behind head on master.

Files with missing lines Patch % Lines
listener/listener.py 66.66% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2763      +/-   ##
==========================================
+ Coverage   64.53%   69.84%   +5.31%     
==========================================
  Files          55       61       +6     
  Lines        6469     7761    +1292     
==========================================
+ Hits         4175     5421    +1246     
- Misses       2294     2340      +46     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Messages semaphore wrapped into handle single consume message so we have better control over semaphore acquires and releases instead of relying on the component that does not know about the semaphore resultion into a single semaphore acquire and release. RHINENG-33634
@pindo696
pindo696 force-pushed the semaphore-release-listener branch from ff08200 to efbffd0 Compare November 19, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants