Skip to content

interface to v2 bucket-oriented query API (queryBuckets / queryBucketsStream) #16

Description

@craigmcchesney

Summary

Add Python client wrappers for the bucket-oriented v2 query RPCs on DpQueryService, complementing the sample-oriented client delivered in #7:

  • queryBuckets() — unary, one page, resumable via nextPageToken
  • queryBucketsStream() — server-streaming, fire-and-consume (no continuation tokens)

Part of the v2 query API epic (#10). #7 intentionally scoped itself to the sample-oriented path (querySamples() / querySamplesStream()) per its own ticket text, and was designed so the bucket methods slot in as an additive follow-on rather than a refactor.

Why this is a clean follow-on (not extra work deferred)

The request side is identical. QueryBucketsRequest wraps the exact same QuerySpec + ExecutionOptions + ResultRepresentation as QuerySamplesRequest. So #7's request-building machinery is reused verbatim:

  • the kind-neutral QueryParams dataclass
  • the PvQuery (PV) / ConfigQuery (CFG) criterion helpers
  • the shared _build_query_spec(params) seam (added in interface to v2 time-series data query API #7 specifically so _build_query_buckets_request() can reuse it)
  • to_timestamp() time handling and page-token paging

The bucket methods also live on the same QueryClient / query channel — no new client class.

The real work: bucket response conversion

The bucket response is substantially richer than the samples ColumnTable, and that's where this issue's effort concentrates:

QueryBucketsResponse
  oneof result: exceptionalResult | bucketQueryResult
  bucketQueryResult:
    dataBuckets: []DataBucket
      pvName, providerId, providerName
      dataTimestamps:            # oneof value:
        samplingClock { startTime, periodNanos, count }   # implicit, regularly-spaced — MUST be expanded
        timestampList { timestamps[] }                    # explicit
      dataValues:                # 16-arm oneof of TYPED COLUMN messages:
        dataColumn | serializedDataColumn |
        doubleColumn | floatColumn | int64Column | int32Column | boolColumn |
        stringColumn | enumColumn(enumId) | imageColumn(imageDescriptor{width,height,channels,encoding}) |
        structColumn(schemaId, bytes) |
        doubleArrayColumn | floatArrayColumn | int32ArrayColumn | int64ArrayColumn | boolArrayColumn
          (array columns carry ArrayDimensions{dims[]})
        each column also carries metadata: ColumnMetadata
    nextPageToken

Key differences from the samples path that drive the work:

  1. samplingClock expansion — a bucket can encode timestamps implicitly as startTime + periodNanos * i for count samples; the conversion layer must expand these into an explicit index.
  2. 16-arm typed-column dataValues oneof — distinct from the samples path's per-value DataValue oneof. Includes typed scalar columns, enum columns (enumId), image columns (imageDescriptor), struct columns (schemaId + bytes), and N-dimensional array columns (ArrayDimensions).
  3. Whole / untrimmed boundary buckets — per the servicer docstring, queryBuckets() returns overlapping boundary buckets intact, so the first/last bucket may contain samples outside the requested [beginTime, endTime). (Contrast: querySamples() trims.) The conversion layer / docs must make this explicit; callers wanting strict trimming use the samples path.
  4. Per-bucket-per-PV assembly — results are a list of DataBuckets (one PV's slice of time each), so assembling a coherent frame means grouping/concatenating across buckets, unlike the samples path's single aligned ColumnTable.

Proposed scope

  • Low-level wrappers: _build_query_buckets_request() (reusing _build_query_spec), unary + streaming _send_*, iter_query_buckets() / iter_query_buckets_stream(), QueryBucketsApiResult.
  • Conversion layer: DataBucket → pandas/NumPy (samplingClock expansion; 16-arm typed-column extraction; documented whole-bucket semantics), reusing the Phase-2 [analysis] optional-extra approach and DataValue/Image handling from interface to v2 time-series data query API #7 where applicable.
  • Unit tests per column arm + both dataTimestamps forms + untrimmed-boundary behavior; integration round-trip vs local :50052.
  • Serialized columns (serializedDataColumn) deferred, same as interface to v2 time-series data query API #7 (fail-loud).

Dependencies

  • Builds on interface to v2 time-series data query API #7 (shared QueryParams / _build_query_spec / QueryClient).
  • Independent of the Sample Status API work; the sampleStatusSelector addition (future) would benefit both sample and bucket paths equally since they share QuerySpec.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions