Skip to content

feat: Add async FDv1 polling data source and feature requester - #475

Open
jsonbailey wants to merge 5 commits into
mainfrom
jb/sdk-2825/async-fdv1-polling
Open

feat: Add async FDv1 polling data source and feature requester#475
jsonbailey wants to merge 5 commits into
mainfrom
jb/sdk-2825/async-fdv1-polling

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Overview

PR 7 of the SDK-60 async epic: the async FDv1 polling data source and feature requester.

  • async_polling.py — async FDv1 polling update processor. Polls the feature requester on an interval and pushes flag/segment data into the data source update sink, updating data source status (VALID / OFF) as appropriate.
  • async_feature_requester.py — async FDv1 feature requester that fetches the full flag/segment payload over HTTP.
  • test_async_polling.py — unit tests for the async polling update processor.

Stacking

This PR is stacked on #464 (base branch jb/sdk-2743/async-fdv1-streaming), which provides the shared datasource_common module these files import. Until #464 merges, this PR will also show #464's commits in its diff; a rebase after #464 merges will drop them, leaving only the three files here.

SDK-2825


Note

Medium Risk
New async data-source path affects client initialization and flag freshness; shutdown and HTTP error handling are security-adjacent but largely mirror existing sync logic with solid test coverage.

Overview
Adds the async FDv1 polling path for the Python SDK: periodic HTTP fetches of flags/segments, store initialization, and data-source status updates aligned with the sync polling processor.

AsyncFeatureRequesterImpl performs GETs to the FDv1 polling endpoint (optional payload filter query param), uses ETag / 304 caching, and only closes its HTTP transport when it created it.

AsyncPollingUpdateProcessor runs polls on AsyncRepeatingTask, writes via sink_or_store, sets ready when the store is initialized, and reports VALID, INTERRUPTED, or OFF (unrecoverable HTTP errors stop polling and unblock init wait). stop() waits for the repeating task to finish before closing the requester so the transport is not torn down mid-request.

AsyncRepeatingTask.wait_stopped() is new so callers can await background task unwind after stop().

Broad unit tests cover requester caching/errors, processor lifecycle, error recovery, sink status, and shutdown ordering.

Reviewed by Cursor Bugbot for commit ef5803c. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey
jsonbailey marked this pull request as ready for review July 30, 2026 22:03
@jsonbailey
jsonbailey requested a review from a team as a code owner July 30, 2026 22:03
Comment thread ldclient/impl/datasource/async_polling.py Outdated
Comment thread ldclient/impl/datasource/async_feature_requester.py
@jsonbailey
jsonbailey force-pushed the jb/sdk-2825/async-fdv1-polling branch from b8b7f52 to 16c4438 Compare July 30, 2026 22:36
Comment thread ldclient/impl/datasource/async_feature_requester.py
Comment thread ldclient/impl/datasource/async_polling.py
Comment thread ldclient/impl/aio/concurrency.py
@jsonbailey
jsonbailey force-pushed the jb/sdk-2825/async-fdv1-polling branch from fff0f5e to 5053397 Compare August 4, 2026 16:32

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5053397. Configure here.


# Signal VALID once the store is populated.
if self._store.initialized and self._data_source_update_sink is not None:
self._data_source_update_sink.update_status(DataSourceState.VALID, None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VALID status requires store initialized

Medium Severity

After a successful poll, async polling only emits DataSourceState.VALID when _store.initialized is true. Sync polling and async streaming update to VALID whenever the sink is present after a successful init. With a custom sink that does not mark the shared store initialized, status can stay non-VALID even though data was delivered.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5053397. Configure here.

Base automatically changed from jb/sdk-2743/async-fdv1-streaming to main August 4, 2026 19:45
Drop the async feature requester's duplicate endpoint definition; use the
shared constant from datasource_common instead.
- Don't set _ready on a generic poll exception, so a transient error during
  startup no longer ends start_wait early (matches sync).
- Close the owned HTTP transport on stop: the feature requester tracks whether
  it created the transport and exposes close(); the polling processor awaits it.
- Drop the dead 'all_data is not None' guard (the requester returns cached data
  on 304, never None) and the fictional None-return polling test.
AsyncRepeatingTask gains wait_stopped() to await the cancelled task; the
polling processor's stop() now waits for the in-flight poll to unwind before
closing the requester's transport, so awaiting stop() guarantees background
work has stopped and the transport isn't closed under a live request.
@jsonbailey
jsonbailey force-pushed the jb/sdk-2825/async-fdv1-polling branch from ca9d5b1 to ef5803c Compare August 4, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants