DataSet, Annotation, and Export clients (issue #6, PR 1)

[craigmcchesney]

Phase 1 of issue #6 — the Python interface to the modernized DataSets / Annotations / Export API. Plan: plan/tickets/6/plan.md.

What's here

Three feature clients on the existing client.annotation facade (plan D1), covering all 10 implemented unary RPCs in the area:

Client	Methods
client.annotation.datasets	save_dataset / get_dataset / query_datasets / iter_datasets / delete_dataset, plus get_datasets(ids) batch fetch (D9)
client.annotation.annotations	save_annotation / get_annotation / query_annotations / iter_annotations / delete_annotation / get_calculations
client.annotation.export	export_data

Alongside them: the DataSetQuery / AnnotationQuery criterion helpers, params and result classes, the data_block() / calculations() / calculations_spec() builders, and the ExportFormat enum. Every unary sender goes through ServiceApiClientBase._dispatch() (#14). patchDataSet / patchAnnotation are not wrapped — reserved "not implemented" placeholders.

Two deliberate differences from the older #5/#9 criterion helpers, both following the proto: attributes(key) accepts an absent values list as a key-only existence search, and criteria is optional because the server treats an empty list as match-all. Back-porting both to the five existing helpers is #40 / #41.

Triage findings folded into the plan

Four corrections were made before implementation, and one more while writing the integration test:

• saveDataSet requires every PV in a data block to already exist in the archive. The error says no PV metadata found for names: [...], but the check is a distinct on pvName over the buckets collection — saved PV metadata does not satisfy it. Neither the proto nor the ticket mentions this. It constrains every future example and test in this area, so it is recorded in both the plan and CLAUDE.md.
• Delete-not-found is a REJECT business error, not a silent success, on both delete methods.
• The server never checks begin < end on a DataBlock — only that each bound is non-zero — so data_block()'s check is the only one there is.
• A calculations SamplingClock is rejected when startTime.epochSeconds is 0, which is a fixture trap worth knowing.
• The redundant sampling_clock() count is kept (D6): deriving it would fork the axis API by caller, and SampleStatusFrame already set the house precedent. Decided now because it is breaking to change later.

Verification

• 570 unit tests (149 new), all passing; ruff lint and format clean; cookbook snippet checker passes.
• 18 integration tests + 12 subtests passing against a live ecosystem built from dp-service fddf692. The test ingests its own samples first (through the generated ingestion stub, since IngestionClient wraps only registerProvider() until Add ingestion API client (full surface: ingestData + streaming) #17) and probes until the bucket is queryable, because ingestData() acks before the data is visible to saveDataSet's check.
• mypy reports the usual protobuf-stub attr-defined errors on the new files, identical in kind to those on the existing shipped clients (the generated stubs carry no type information). Not in CI.

Not in this PR

Per the plan's two-PR split (Q9), PR 2 carries Phases 2–4: data_frame.py builders and the pure-Python read side, the pandas bridges under [analysis], the datasets-and-annotations.md cookbook recipe, and the README / CLAUDE.md usage section. The ticket stays open until that merges.

🤖 Generated with Claude Code

https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn

[Copilot]

🟡 Changes recommended

A newly added docstring in calculations() references a data_frame.data_frame() builder that is not present in this PR, which is misleading for current users.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds Phase 1 Python client support for the modernized Annotation Service DataSets / Annotations / Export APIs (issue #6) by introducing three new feature clients under the existing client.annotation facade, plus the associated builders/helpers and comprehensive unit + integration tests.

Changes:

• Introduce DataSetClient, AnnotationsClient, and ExportClient (plus criterion helpers, params/results, and builders like data_block(), calculations(), calculations_spec() and ExportFormat).
• Wire the new feature clients into the AnnotationClient facade and re-export the new public API surface from dp_python_lib.client.
• Add extensive unit tests and a live-ecosystem integration test covering dataset/annotation/calculations round-trips and key server behaviors.

File summaries

File	Description
tests/unit/test_export_client.py	Unit coverage for ExportClient, ExportFormat, calculations_spec(), params validation, and _dispatch error tiers.
tests/unit/test_dataset_client.py	Unit coverage for DataSetClient, DataSetQuery, data_block(), paging, and batch get_datasets().
tests/unit/test_annotations_client.py	Unit coverage for AnnotationsClient, AnnotationQuery, calculations(), and CRUD/paging/error handling.
tests/unit/test_annotation_client.py	Pins facade wiring to ensure all feature clients are exposed and share the channel correctly.
tests/integration/test_datasets_annotations_integration.py	End-to-end integration test (requires running services) covering dataset/annotation/calculations workflows and delete semantics.
src/dp_python_lib/client/export_client.py	New export client + output-format enum + export request params/result wrappers.
src/dp_python_lib/client/dataset_client.py	New dataset client + criteria helpers + data_block() builder + paging and batch fetch utilities.
src/dp_python_lib/client/annotations_client.py	New annotations/calculations client + criteria helpers + calculations() builder and CRUD/paging.
src/dp_python_lib/client/annotation_client.py	Extend facade to expose .datasets, .annotations, and .export.
src/dp_python_lib/client/init.py	Re-export new clients/helpers/results for the public API surface.
plan/tickets/6/plan.md	Plan updates reflecting completed Phase 1 work and triage findings.
CLAUDE.md	Documentation updates to include the new clients and key invariants/behaviors.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.