feat(api): probe a provider with the request's key and keep nothing

CHANGELOG.md

@@ -64,6 +64,7 @@
 
 ### Added
 
+- `POST /providers/{provider}/probe` tells what a key can reach, per dataset, without keeping it (#802). The key comes in the `X-Provider-Key` header and is the only one used — a stored credential or one in the server's environment is never tried, in either deployment mode. Each dataset of the provider that kpubdata has a spec for gets one of kpubdata's probe statuses (`available`, `application_required`, `params_invalid` …) and the provider service it belongs to. Nothing is stored. After 45 seconds no further dataset is started; those not reached are named in `not_probed`. Contract 1.87.0.
 - An upload says when it expires (#1047, API contract 1.86.0, additive). #1045 deletes an upload once it is older than a multi-user deployment's retention period, and its owner had no way to see that coming: the metadata carried `created_at` and the period is a server setting. `POST /uploads` and `GET /uploads/{upload_id}` now carry `expires_at` — `created_at` plus the period in force, the same boundary the clean-up uses — and `null` when nothing will delete the upload (a single-user deployment, or retention turned off).
 - A build can say which run it retries (#1042, decided in kpubdata#812, API contract 1.85.0, additive). A `run_id` is one attempt and is not reused once it ended, so a retry is a new run: `retry_of` on `POST /build` and `POST /builds` names the earlier one, and it is shown on the job (`BuildJob.retry_of`, also after a restart) and written to the manifest. The named run must be one the caller may read — otherwise the answer is the one reading it would give — and cannot be the run itself. The refusal of a run id that already ended now carries `code: run_id_ended`.
 - Per-user upload limits in a multi-user deployment (#1045, decided in kpubdata#812, API contract 1.84.0). Nothing bounded what one account could upload, and an upload was kept until its owner deleted it. Each owner may now hold 50 files and 1 GiB in total — an upload past either is refused with 409 `upload_quota_exceeded`, saying which limit and what is in use — and an upload older than 30 days is deleted, when the service starts and when its owner next uploads. `KPUBDATA_BUILDER_UPLOAD_MAX_FILES`, `KPUBDATA_BUILDER_UPLOAD_MAX_TOTAL_BYTES` and `KPUBDATA_BUILDER_UPLOAD_RETENTION_DAYS` override them; `0` turns one off. A single-user deployment applies none. **Retention deletes data:** a saved spec that names an upload past 30 days no longer builds until the file is uploaded again.

contract/builder-api.yaml

@@ -1,7 +1,7 @@
 openapi: 3.1.0
 info:
   title: KPubData Builder Service API
-  version: "1.86.0"
+  version: "1.87.0"
   description: >-
     KPubData Builder(패키지 `kpubdata-builder`) 서비스 계약. BuildSpec 검증, preview, build 실행, artifact 조회,
     계약 버전 조회를 제공한다. CLI와 HTTP service mode(#36)가 동일한 도메인 계약을
@@ -20,6 +20,8 @@ info:
     추가한다(#504, additive). v1.8.0은 stable principal별 encrypted Provider
     credential CRUD와 Provider status/test API를 추가한다(#492, additive). v1.9.0은
     `/query` 응답에 child startup과 Polars engine 실행 시간을 추가한다(#523, additive).
+    v1.87.0 adds `POST /providers/{provider}/probe` (#802, additive): what the key in the request's `X-Provider-Key` header can reach, per source dataset, in kpubdata's probe vocabulary (`available`, `application_required`, `params_invalid` …) with the provider service each dataset belongs to. Only the request's key is used — a stored credential or one in the server's environment is never tried, in either deployment mode — and nothing is stored. Without the header the answer is 400 `provider_key_required`. One call is made per dataset; after 45 seconds no further dataset is started, and those not reached are named in `not_probed` with `complete: false`.
+
     v1.86.0 says when an upload expires (#1047, additive): `UploadMetadata.expires_at`, `created_at` plus the retention period in force, or `null` when nothing will delete the upload.
     v1.85.0 lets a build point at the run it retries (#1042, additive): `retry_of` on `POST /build` and `POST /builds`, shown on `BuildJob.retry_of` and in the manifest. The refusal of a run id that already ended carries `code: run_id_ended`, and both routes name it as an example.
     v1.84.0 adds per-user upload limits in a multi-user deployment (#1045, additive): `POST /uploads` answers 409 `upload_quota_exceeded` when the owner's file count or total size would pass the limit, and uploads past the retention period are deleted.
@@ -381,6 +383,68 @@ paths:
         "502":
           description: 런타임 Provider 카탈로그 조회 실패
 
+  /providers/{provider}/probe:
+    post:
+      operationId: probeProviderKey
+      summary: 요청에 실린 키로 무엇이 되는지 데이터셋별로 확인 (#802)
+      description: >-
+        `X-Provider-Key` 헤더의 키만 쓴다. 저장된 credential 이나 서버 환경변수의 키는
+        배포 모드와 무관하게 쓰지 않는다. 키도 결과도 저장하지 않으며, 응답에는 키에서
+        유도한 값이 없다. 데이터셋마다 한 번 호출한다(15초, 재시도·캐시 없음). 45초가
+        지나면 새 호출을 시작하지 않고, 닿지 못한 데이터셋은 `not_probed` 에 적는다.
+        replay 모드의 Builder(데모·개발용)는 provider 를 호출하지 않고 기록된 응답을
+        재생하므로, 어떤 키에도 `available` 로 답한다 — 키에 대해 아무것도 말하지 않는다.
+      parameters:
+        - $ref: "#/components/parameters/ProviderKey"
+        - $ref: "#/components/parameters/ProviderName"
+      requestBody:
+        required: false
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/ProviderProbeRequest"
+      responses:
+        "200":
+          description: 데이터셋별 probe 결과
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ProviderProbeResponse"
+              examples:
+                Probed:
+                  summary: 하나는 바로 쓸 수 있고 하나는 활용신청이 필요함
+                  value:
+                    provider: datago
+                    probed_at: "2026-10-06T09:30:00+00:00"
+                    complete: true
+                    datasets:
+                      - dataset: apt_trade
+                        service_id: RTMSDataSvcAptTradeDev
+                        status: available
+                        detail: ""
+                        http_status: 200
+                      - dataset: apt_rent
+                        service_id: RTMSDataSvcAptRent
+                        status: application_required
+                        detail: "SERVICE_KEY_IS_NOT_REGISTERED_ERROR"
+                        http_status: 200
+                    not_probed: []
+        "400":
+          description: >-
+            `provider_key_required` — 이 provider 의 키가 `X-Provider-Key` 헤더에 없다.
+            `invalid_request` — body 가 `datasets` 외의 필드를 갖거나, `datasets` 가 비었거나
+            probe 할 수 없는 데이터셋을 가리킨다. `invalid_provider_key` — 헤더 형식 오류.
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/Error"
+        "401":
+          $ref: "#/components/responses/Unauthorized"
+        "404":
+          description: 알려지지 않은 Provider
+        "502":
+          description: 런타임 Provider 카탈로그 조회 실패, 또는 probe 를 실행할 수 없음
+
   /providers/{provider}/credential:
     get:
       operationId: getProviderCredential
@@ -6983,6 +7047,71 @@ components:
           maximum: 599
           description: kpubdata가 구조적으로 신뢰 가능한 status를 제공한 경우에만 포함
 
+    ProviderProbeRequest:
+      type: object
+      additionalProperties: false
+      properties:
+        datasets:
+          type: array
+          minItems: 1
+          maxItems: 50
+          items:
+            type: string
+          description: >-
+            Dataset names within the provider (as `GET /catalog` lists them). Omitted:
+            every dataset of the provider that can be probed.
+
+    ProviderProbeResponse:
+      type: object
+      required: [provider, probed_at, complete, datasets, not_probed]
+      properties:
+        provider:
+          type: string
+        probed_at:
+          type: string
+          format: date-time
+        complete:
+          type: boolean
+          description: False when `not_probed` names a dataset.
+        datasets:
+          type: array
+          items:
+            $ref: "#/components/schemas/ProviderProbeDataset"
+        not_probed:
+          type: array
+          items:
+            type: string
+          description: >-
+            Datasets nothing was observed about: the time budget ran out before their
+            call, or they have no spec to probe with.
+
+    ProviderProbeDataset:
+      type: object
+      required: [dataset, service_id, status, detail, http_status]
+      properties:
+        dataset:
+          type: string
+          description: Dataset name within the provider.
+        service_id:
+          type: string
+          description: >-
+            The provider service the dataset belongs to. An application at the provider
+            is granted per service, so datasets of one service share a verdict.
+        status:
+          type: string
+          description: >-
+            One of kpubdata's `PROBE_STATUSES` (ADR 0005 there): `available`,
+            `auth_unknown`, `application_required`, `params_invalid`, `rate_limited`,
+            `temporarily_unavailable`, `network_error`, `insufficient_metadata`,
+            `retired`. Not an enum here: a kpubdata release may add one, and a client
+            should show an unknown status as it is.
+        detail:
+          type: string
+          description: What the provider said, with no key in it. May be empty.
+        http_status:
+          type: [integer, "null"]
+          description: The HTTP status of the one call, where it was observed.
+
     StructuredProblem:
       type: object
       required: [code, path, message]

contract/fixtures/responses.json

@@ -1,6 +1,6 @@
 {
   "fixture_format": 1,
-  "contract_version": "1.86.0",
+  "contract_version": "1.87.0",
   "probe_field": "future_optional_field",
   "rules": "A response may gain optional fields in any minor version; a client ignores fields it does not know. A required field keeps its name and type until the next major version; a client rejects a body whose required field is missing or mistyped.",
   "fixtures": [
@@ -407,6 +407,88 @@
       },
       "broken_path": "$.provider"
     },
+    {
+      "operation_id": "probeProviderKey",
+      "method": "POST",
+      "path": "/providers/{provider}/probe",
+      "status": 200,
+      "example": "Probed",
+      "current": {
+        "provider": "datago",
+        "probed_at": "2026-10-06T09:30:00+00:00",
+        "complete": true,
+        "datasets": [
+          {
+            "dataset": "apt_trade",
+            "service_id": "RTMSDataSvcAptTradeDev",
+            "status": "available",
+            "detail": "",
+            "http_status": 200
+          },
+          {
+            "dataset": "apt_rent",
+            "service_id": "RTMSDataSvcAptRent",
+            "status": "application_required",
+            "detail": "SERVICE_KEY_IS_NOT_REGISTERED_ERROR",
+            "http_status": 200
+          }
+        ],
+        "not_probed": []
+      },
+      "with_additive_fields": {
+        "provider": "datago",
+        "probed_at": "2026-10-06T09:30:00+00:00",
+        "complete": true,
+        "datasets": [
+          {
+            "dataset": "apt_trade",
+            "service_id": "RTMSDataSvcAptTradeDev",
+            "status": "available",
+            "detail": "",
+            "http_status": 200,
+            "future_optional_field": "added by a later minor contract version"
+          },
+          {
+            "dataset": "apt_rent",
+            "service_id": "RTMSDataSvcAptRent",
+            "status": "application_required",
+            "detail": "SERVICE_KEY_IS_NOT_REGISTERED_ERROR",
+            "http_status": 200,
+            "future_optional_field": "added by a later minor contract version"
+          }
+        ],
+        "not_probed": [],
+        "future_optional_field": "added by a later minor contract version"
+      },
+      "additive_paths": [
+        "$.datasets[0]",
+        "$.datasets[1]",
+        "$"
+      ],
+      "required_type_broken": {
+        "provider": 12345,
+        "probed_at": "2026-10-06T09:30:00+00:00",
+        "complete": true,
+        "datasets": [
+          {
+            "dataset": "apt_trade",
+            "service_id": "RTMSDataSvcAptTradeDev",
+            "status": "available",
+            "detail": "",
+            "http_status": 200
+          },
+          {
+            "dataset": "apt_rent",
+            "service_id": "RTMSDataSvcAptRent",
+            "status": "application_required",
+            "detail": "SERVICE_KEY_IS_NOT_REGISTERED_ERROR",
+            "http_status": 200
+          }
+        ],
+        "not_probed": []
+      },
+      "broken_path": "$.provider"
+    },
     {
       "operation_id": "getProviderCredential",
       "method": "GET",

src/kpubdata_builder/service/app.py

@@ -51,7 +51,7 @@
 from . import datasets as datasets_service
 from . import monitoring as monitoring_service
 from . import ownership as ownership_module
-from . import pii_reads, publish_credentials, request_credentials
+from . import pii_reads, provider_probe, publish_credentials, request_credentials
 from . import publish as publish_service
 from .analyses_api import AnalysesApiService, AnalysisStore
 from .auth import AuthError, Principal, authenticate
@@ -419,7 +419,9 @@ def _enforce_ownership() -> bool:
 #   used-run-id refusal carries code run_id_ended (#1042, additive).
 # 1.85.0 -> 1.86.0: UploadMetadata.expires_at — when the retention period deletes the
 #   upload, or null (#1047, additive).
-API_CONTRACT_VERSION = "1.86.0"
+# 1.86.0 -> 1.87.0: POST /providers/{provider}/probe — what the key in the request's
+#   X-Provider-Key header can reach, per dataset; nothing is stored (#802, additive).
+API_CONTRACT_VERSION = "1.87.0"
 
 #: How long a synchronous ``POST /build`` waits for a build slot before it answers
 #: ``build_queue_full`` (#1040). Long enough to ride out a short build ahead of it, short
@@ -450,6 +452,8 @@ def __init__(
         upload_repository: UploadRepository | None = None,
         provider_test_operation: ProviderTestOperation = default_provider_test,
         provider_test_timeout: float | None = None,
+        open_probe: provider_probe.OpenProbe = provider_probe.open_kpubdata_probe,
+        probe_datasets: provider_probe.ListDatasets = provider_probe.spec_dataset_ids,
         async_max_workers: int = 10,
         async_max_queue_size: int = 10,
         max_concurrent_builds: int | None = None,
@@ -523,6 +527,8 @@ def __init__(
             provider_test_operation=self._provider_test_operation,
             provider_test_timeout=self._provider_test_timeout,
             test_log=lambda: self._provider_tests(),
+            open_probe=open_probe,
+            probe_datasets=probe_datasets,
         )
         # Upload repository initialized only when needed (#498) — pass lambda,
         # not property value directly (#498) — calling property on every request
@@ -813,6 +819,16 @@ def provider_status(self, provider: str, *, principal: Principal) -> ServiceResp
         """Perform lightweight connection test with current principal's credential."""
         return self._providers_service.provider_status(provider, principal=principal)
 
+    def probe_provider(
+        self,
+        provider: str,
+        body: Mapping[str, JsonValue] | None,
+        *,
+        principal: Principal,
+    ) -> ServiceResponse:
+        """Say what the key this request carries can reach, per dataset (#802)."""
+        return self._providers_service.probe_provider(provider, body, principal=principal)
+
     def provider_credential(self, provider: str, *, principal: Principal) -> ServiceResponse:
         """Return saved credential metadata for current principal without plaintext."""
         return self._providers_service.provider_credential(provider, principal=principal)