Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
iter_export_pagesto bothProoflineClientandAsyncProoflineClient. It reads one export window end to end by following the signedX-Proofline-Export-Next-Cursorcontinuations and yields everyExportPage:Who it helps
Today every export user hand-rolls the loop the README describes: "Repeat with the returned cursor and the same dataset, window, format and grouping." That loop is easy to get wrong — a continuation sent with a different
group_by, a window that drifts between pages, or a dropped cursor. The helper pinsdataset,from/to,formatandgroup_byacross all requests and advances onlycursor, so pullingreviews,findings,spend,finding_outcomesorfinding_validationsbecomes a singlefor/async forover CSV or NDJSON pages. It targets the same audience as the existingiter_reviews: scripts and pipelines that want the whole window instead of one page.Decisions
ExportPage, not bare rows. Each page keeps its rawtext, decodedrows,settled_untilhorizon andnext_cursor, so callers can stream per-page CSV (the contract repeats the header on every page), append NDJSON, or flatten rows themselves while still keeping the checkpoint horizon.iter_reviews.max_pagesdefaults to 100 (1..1000; a page holds at most 5000 rows per the contract), raisesValueErroroutside that range, and raisesProoflineError(urn:proofline:sdk:pagination-limit, detailExport pagination reached the configured bound.) when a continuation is still offered at the bound._pagination_limitgained asubjectargument so the export message does not read as a review failure; existing review callers keep the old text.cursor=— raisesurn:proofline:sdk:invalid-paginationwith the same detail style as_next_cursor.settled_untilenforcement. The contract documents the horizon as frozen for the export but does not promise identical values across separate continuations, so the helper reports each page's horizon instead of failing on a mismatch.scripts/contracts.pynow listsiter_export_pagesin thehelperswhitelist next toiter_reviewsandwait_for_review. The check still requires both facades to expose exactly the 21 generated operations plusexport_reviews, so the operation surface is unchanged;openapi.public.jsonandproofline/_generated/are untouched.next()/__anext__(), the same wayiter_reviewsvalidatespage_size,max_pagesandmax_items.Verification
Run from a clean checkout:
That is the entry point of the
Checksworkflow in.github/workflows/ci.yml. It bootstraps the pinned tools fromrequirements-dev.txtand runsscripts/generate.py --check,scripts/contracts.py,ruff check .,ruff format --check .,mypy proofline scripts,pytest -q,python -m build(one wheel plus one sdist) and the install smoke test against a local HTTP peer.Result on this branch: every step passed —
All checks passed!(ruff check and format),Success: no issues found on 225 source files(mypy),Python operation and outcome-vocabulary contracts passed.,91 passed(pytest),Installed Python SDK sync/async HTTP smoke passed., andPython SDK generation, HTTP contracts, wheel and source installs passed.(exit 0, two runs: once after the code changes and once after the README addition).New test:
tests/test_http.py::test_export_iteration_keeps_the_window_and_refuses_cycles_or_bounds, parametrized over sync and async, driving a real loopback HTTP peer. It asserts that:from,to,formatandgroup_bystay identical on every request while onlycursoradvances;urn:proofline:sdk:invalid-paginationafter two requests;max_pages=2raisesProoflineErrorwith the export bound detail after two requests;max_pages=0andmax_pages=1001raiseValueErrorbefore any request is sent.Focused run:
python -m pytest -q -k export→4 passed. Full suite before the change also passed (baselinepython3 scripts/check.pyon a clean checkout, exit 0).Files changed:
proofline/client.py(feature),tests/test_http.py(tests),scripts/contracts.py(helper whitelist),README.md(usage next to the existing export example).