diff --git a/CHANGELOG.md b/CHANGELOG.md index 69703325..1f40b950 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,7 @@ ### Changed +- The kpubdata pin moves to the 0.9 line (`kpubdata>=0.9.0,<0.10`, #990, #1002, kpubdata#780). 0.9.0 is the first release with `Client(env_keys=False)`: on 0.8.0 `serve` refused to start in a multi-user deployment because the operator's keys could not be kept out of a request, and now it starts. Seven imports from kpubdata's private modules — `find_spec`, `discover_specs`, `KPubDataConfig` and `SENSITIVE_PARAM_KEYS` — use the public names, and their allowlist entries are gone. **What changes for a deployment:** kpubdata 0.9.0 declares terms for more datasets, and Builder acts on them — `datago.air_quality` is now `forbidden` (no publish, and no rows through query, preview, exports or downloads), and the 16 specs whose `allowed` was never confirmed are `unknown` (no public publish). kpubdata's own breaking changes apply too: HTTP 401 is `AuthError`, a total count of zero is `0`, NODATA is an empty result. - The automated deployment to the `app-01` VM is removed. `.github/workflows/deploy.yml` ran on every push to `main` that touched the service, built and pushed an image, then failed at its SSH step — the deployment it was written for is not being used. The workflow and `ops/deploy/app-01-rollout.sh`, which only it ran, are deleted. `docker.yml` still publishes the images to GHCR, and `docker-compose.prod.app.yml` is unchanged for anyone who runs it by hand. `docs/deploy.md`, `docs/CREDENTIAL_SURFACE.md` and ADR 0017 say so. - A pull request builds the documentation site in strict mode. `mkdocs build --strict` ran only in `docs.yml`, on a push to `main`, so the link #1026 fixed was found after it had merged and the deploy had failed. `ci.yml` has a `Docs build (strict)` job now and the `CI gate` waits for it; `docs.yml` still does the deploy. `tests/unit/test_ci_docs_gate.py` holds that the job exists, that the gate needs it and that it runs the command the deploy runs. - The contract declares four things the service already sent (#994, API contract 1.80.0, description only). `X-Provider-Key` is a header parameter of the five operations that call a provider (`previewBuild`, `createBuild`, `submitBuild`, `getProviderStatus`, `testProviderConnection`) — it was mentioned only in `info.description`, so a client's drift test had nothing to check it against. The 429 `auth_throttled` and the overload 503 are shared responses any operation can answer (`components.responses.AuthThrottled`, `ServerOverloaded`), and `X-Request-ID` is `components.headers.RequestId`. `tests/unit/test_contract_shared_declarations.py` compares each with what the code sends. Generating the route table from the route adapter, the remaining part of #994, is not done. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e84633b5..40900d6f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -102,11 +102,11 @@ CI(`publish-dataset.yml`)와 패키지 배포 환경에서는 `--no-sources` 플 uv sync --extra dev --extra publish --no-sources ``` -`--no-sources`는 `[tool.uv.sources]`를 무시하고, `pyproject.toml`의 `dependencies`에 명시된 **PyPI 릴리스 핀**(`kpubdata>=0.8.0,<0.9`)을 직접 설치합니다. 형제 디렉터리가 없어도 작동합니다. +`--no-sources`는 `[tool.uv.sources]`를 무시하고, `pyproject.toml`의 `dependencies`에 명시된 **PyPI 릴리스 핀**(`kpubdata>=0.9.0,<0.10`)을 직접 설치합니다. 형제 디렉터리가 없어도 작동합니다. -#### 핀 범위(`>=0.8.0,<0.9`)를 이렇게 설정한 이유 +#### 핀 범위(`>=0.9.0,<0.10`)를 이렇게 설정한 이유 -`kpubdata-builder`는 `kpubdata`의 공개 API만 씁니다(kpubdata ADR 0007, `scripts/check_kpubdata_imports.py`). kpubdata 0.8.0은 코드 열의 앞자리 0을 보존해 `str`로 돌려주는 breaking 릴리스이며(kpubdata#613), #702·#688·#689가 기다리던 라이선스·열 메타데이터 필드를 더합니다. 0.9 이상은 검증 전이라 허용하지 않습니다 (관련 이슈: #213). CI 의 `kpubdata <버전>` 잡이 범위 안의 모든 릴리스로 스위트를 돌립니다(#832). +`kpubdata-builder`는 `kpubdata`의 공개 API만 씁니다(kpubdata ADR 0007, `scripts/check_kpubdata_imports.py`). kpubdata 0.9.0은 `Client(env_keys=False)` 가 들어간 첫 릴리스입니다 — 다중 사용자 모드가 운영자 키를 요청에서 떼어 놓는 데 필요합니다(#990, kpubdata#780). breaking 릴리스이기도 합니다: 모르는 `Client` 인자를 거부하고, HTTP 401 이 `AuthError` 가 되고, 건수 0 을 `None` 이 아니라 `0` 으로 보고하고, NODATA 가 모든 경로에서 빈 결과입니다. 그래서 0.9 라인에 고정합니다. 핀의 정본은 `pyproject.toml`의 `dependencies`입니다. 이 표와 어긋나면 `pyproject.toml`이 맞습니다. @@ -115,7 +115,7 @@ uv sync --extra dev --extra publish --no-sources | 환경 | 명령 | `kpubdata` 소스 | | :--- | :--- | :--- | | 로컬 개발 | `uv sync --extra dev` | `../kpubdata` (editable, 형제 디렉터리 필요) | -| CI / 배포 | `uv sync ... --no-sources` | PyPI (`kpubdata>=0.8.0,<0.9`) | +| CI / 배포 | `uv sync ... --no-sources` | PyPI (`kpubdata>=0.9.0,<0.10`) | ## 3. 브랜치 전략과 협업 규칙 diff --git a/pyproject.toml b/pyproject.toml index 9fdf5b11..23efe0c0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -28,11 +28,12 @@ classifiers = [ ] dependencies = [ # Builder depends on the kpubdata public API only (ADR 0007 in kpubdata, - # scripts/check_kpubdata_imports.py). 0.8.0 is breaking: code columns keep their - # leading zeros and come back as str (kpubdata#613), and it adds the licence and - # column-metadata fields #702, #688 and #689 wait on. Pin to the 0.8 line; compat - # policy: #213. - "kpubdata>=0.8.0,<0.9", + # scripts/check_kpubdata_imports.py). 0.9.0 is the first release with + # `Client(env_keys=False)`, which the multi-user mode needs to keep the operator's + # keys out of a request (#990, kpubdata#780), and it is breaking: unknown `Client` + # arguments are refused, HTTP 401 is `AuthError`, a total count of zero is `0`, and + # NODATA is an empty result on every path. Pin to the 0.9 line; compat policy: #213. + "kpubdata>=0.9.0,<0.10", "cryptography>=50,<51", # The tabular engine the DuckDB migration moves to (ADR 0021, #866). 1.2.0 is the first # release with every sandbox setting the query profile needs — allowed_paths, diff --git a/scripts/check_kpubdata_imports.py b/scripts/check_kpubdata_imports.py index aadba0aa..8b5af53b 100644 --- a/scripts/check_kpubdata_imports.py +++ b/scripts/check_kpubdata_imports.py @@ -62,31 +62,9 @@ "and transport; kpubdata has no public verify API (kpubdata#667). Signatures are " "pinned by tests/unit/test_kpubdata_internal_surface.py." ) -_SPEC_LOOKUP_REASON = ( - "Looks up a SpecDefinition by id; the public Client returns DatasetRef, not the " - "spec model verify needs (kpubdata#667)." -) -_CONFIG_REASON = ( - "Reads the operator's provider key the way kpubdata resolves it from the " - "environment; Client does not expose key resolution publicly (kpubdata#667)." -) #: (path, module, name) -> reason. ``name`` is "" for ``import module``. ALLOWLIST: dict[tuple[str, str, str], str] = { - ( - "src/kpubdata_builder/logging_redaction.py", - "kpubdata.transport._sensitive", - "SENSITIVE_PARAM_KEYS", - ): ( - "The one canonical list of credential parameter names; Builder keeps no copy " - "so the two cannot drift. Needs a public export (kpubdata#667)." - ), - ("src/kpubdata_builder/service/providers.py", "kpubdata.config", "KPubDataConfig"): ( - _CONFIG_REASON - ), - ("src/kpubdata_builder/verify/runner.py", "kpubdata.config", "KPubDataConfig"): ( - _VERIFY_REASON - ), ("src/kpubdata_builder/verify/runner.py", "kpubdata.core.executor", "SpecExecutor"): ( _VERIFY_REASON ), @@ -112,14 +90,6 @@ ("src/kpubdata_builder/verify/runner.py", "kpubdata.transport.http", "HttpTransport"): ( _VERIFY_REASON ), - ("src/kpubdata_builder/agent/monitor.py", "kpubdata.core.spec", "find_spec"): ( - _SPEC_LOOKUP_REASON - ), - ("src/kpubdata_builder/agent/pipeline.py", "kpubdata.core.spec", "find_spec"): ( - _SPEC_LOOKUP_REASON - ), - ("src/kpubdata_builder/cli.py", "kpubdata.core.spec", "find_spec"): _SPEC_LOOKUP_REASON, - ("src/kpubdata_builder/cli.py", "kpubdata.core.spec", "discover_specs"): (_SPEC_LOOKUP_REASON), } diff --git a/src/kpubdata_builder/agent/monitor.py b/src/kpubdata_builder/agent/monitor.py index 3f94c6af..5b98a54f 100644 --- a/src/kpubdata_builder/agent/monitor.py +++ b/src/kpubdata_builder/agent/monitor.py @@ -97,7 +97,7 @@ def check_approval(dataset_id: str, *, api_key: str | None = None) -> str: Returns: Status string: HEALTHY, NEEDS_APPLICATION, INVALID_KEY, etc. """ - from kpubdata.core.spec import find_spec + from kpubdata import find_spec from kpubdata_builder.verify import verify_dataset diff --git a/src/kpubdata_builder/agent/pipeline.py b/src/kpubdata_builder/agent/pipeline.py index 4ac20638..e0492149 100644 --- a/src/kpubdata_builder/agent/pipeline.py +++ b/src/kpubdata_builder/agent/pipeline.py @@ -81,7 +81,7 @@ def run_pipeline( [ sys.executable, "-c", - "import sys; from kpubdata.core.spec import find_spec; " + "import sys; from kpubdata import find_spec; " "sys.exit(0 if find_spec(sys.argv[1]) is not None else 1)", dataset_id, ], diff --git a/src/kpubdata_builder/cli.py b/src/kpubdata_builder/cli.py index 1a5d326c..2cb536f7 100644 --- a/src/kpubdata_builder/cli.py +++ b/src/kpubdata_builder/cli.py @@ -474,7 +474,7 @@ def build_parser() -> argparse.ArgumentParser: #: Why a client that must not use the operator's keys cannot be built (#990). ENV_KEYS_UNSUPPORTED = ( "the installed kpubdata cannot keep the environment's provider keys out of a client " - "(Client has no env_keys option; it was added after 0.8.0), so a request without its " + "(Client has no env_keys option; it came with 0.9.0), so a request without its " "own key would run on the operator's. Install a kpubdata release that has it, or " "turn KPUBDATA_BUILDER_REQUIRE_OWN_PROVIDER_CREDENTIAL and multi-user mode off." ) @@ -485,7 +485,8 @@ def client_keeps_environment_keys_out() -> bool: kpubdata 0.8.0 has no such option and accepts any keyword without an error, so passing ``env_keys=False`` to it does nothing and says nothing. Asking the signature - is the only way to know the option is honoured. + is the only way to know the option is honoured. Builder's pin is the 0.9 line, which + has it; the check stays for an environment that installed something older anyway. """ import inspect @@ -1347,7 +1348,7 @@ def _run_verify( page_size: Records per test request. hashes_path: Optional path to previous schema hashes YAML. """ - from kpubdata.core.spec import discover_specs, find_spec + from kpubdata import discover_specs, find_spec from .verify import runner as _verify_runner diff --git a/src/kpubdata_builder/logging_redaction.py b/src/kpubdata_builder/logging_redaction.py index 72bb48c1..b5582fe1 100644 --- a/src/kpubdata_builder/logging_redaction.py +++ b/src/kpubdata_builder/logging_redaction.py @@ -28,7 +28,7 @@ from collections.abc import Iterable from urllib.parse import quote, quote_plus -from kpubdata.transport._sensitive import SENSITIVE_PARAM_KEYS +from kpubdata import SENSITIVE_PARAM_KEYS REDACTED = "[REDACTED]" diff --git a/src/kpubdata_builder/service/providers.py b/src/kpubdata_builder/service/providers.py index 4f963b9d..5d1f0d66 100644 --- a/src/kpubdata_builder/service/providers.py +++ b/src/kpubdata_builder/service/providers.py @@ -11,8 +11,7 @@ from datetime import datetime, timezone from typing import Literal, Protocol, cast -from kpubdata import Client, Operation -from kpubdata.config import KPubDataConfig +from kpubdata import Client, KPubDataConfig, Operation from kpubdata.core.models import DatasetRef from kpubdata.exceptions import ( AuthError, diff --git a/src/kpubdata_builder/verify/runner.py b/src/kpubdata_builder/verify/runner.py index e205a5a6..c10ebae0 100644 --- a/src/kpubdata_builder/verify/runner.py +++ b/src/kpubdata_builder/verify/runner.py @@ -119,7 +119,7 @@ def verify_dataset( # ``tests/unit/test_kpubdata_internal_surface.py`` pins this list, # so upgrade breaks in CI, not at runtime. When starting to use new internal symbols, # add them to that list too. - from kpubdata.config import KPubDataConfig + from kpubdata import KPubDataConfig from kpubdata.core.executor import ( SpecExecutor, check_payload_error, diff --git a/tests/unit/test_verify_cli.py b/tests/unit/test_verify_cli.py index 8b4bde5a..13d44a2c 100644 --- a/tests/unit/test_verify_cli.py +++ b/tests/unit/test_verify_cli.py @@ -12,7 +12,7 @@ # All tests in this module patch the verify.runner module which is imported # lazily inside _run_verify via ``from .verify import runner as _verify_runner``. _RUNNER_MOD = "kpubdata_builder.verify.runner" -_SPEC_MOD = "kpubdata.core.spec" +_SPEC_MOD = "kpubdata" class TestVerifyParser: diff --git a/uv.lock b/uv.lock index 6d267c90..a01d4b58 100644 --- a/uv.lock +++ b/uv.lock @@ -842,16 +842,16 @@ wheels = [ [[package]] name = "kpubdata" -version = "0.8.0" +version = "0.9.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "httpx" }, { name = "pyyaml" }, { name = "typing-extensions", marker = "python_full_version < '3.12'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/71/01/14dc8f01545ecdc3f228cd1cb1b4f8a512a623058dfe2f9798cb6e118cdd/kpubdata-0.8.0.tar.gz", hash = "sha256:3598b81edc37ee2605d41c4c80091c950a0131116307ad103dea5dfcb289b194", size = 1312246, upload-time = "2026-09-30T10:16:13.216Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/37/3df78daeb7d361942e7220d0afe226bb651d8c7a2ed0b101ce0ace2157db/kpubdata-0.9.0.tar.gz", hash = "sha256:18f71e9c7cf075ddf98df64ac9b84733d6dda1df607ad43069c140d48b849dac", size = 1444287, upload-time = "2026-10-06T02:47:42.667Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/80/82/a2bdd240b508d895c8b1e5190b4eb7a58cf403eacc538bc5a7554bd69c0c/kpubdata-0.8.0-py3-none-any.whl", hash = "sha256:00f4ff825613991a4406f0e6c3f2ebb8bce4736fc6609238f6d3b8c333391f5f", size = 220492, upload-time = "2026-09-30T10:16:11.675Z" }, + { url = "https://files.pythonhosted.org/packages/53/15/f6d923459bb91049b68254a761e9a1fb052894620e1e5f3ab5b4f57ee0e1/kpubdata-0.9.0-py3-none-any.whl", hash = "sha256:bd42e41e803df5d00d3cd0d96d63c5e724d0a30af49cf612c6093c1b9406c21f", size = 239629, upload-time = "2026-10-06T02:47:41.171Z" }, ] [[package]] @@ -912,7 +912,7 @@ requires-dist = [ { name = "huggingface-hub", marker = "extra == 'huggingface'", specifier = ">=0.23,<3" }, { name = "huggingface-hub", marker = "extra == 'publish'", specifier = ">=0.23,<3" }, { name = "kaggle", marker = "extra == 'publish'", specifier = ">=1.6,<2" }, - { name = "kpubdata", specifier = ">=0.8.0,<0.9" }, + { name = "kpubdata", specifier = ">=0.9.0,<0.10" }, { name = "mkdocs", marker = "extra == 'docs'", specifier = ">=1.6,<2" }, { name = "mkdocs-material", marker = "extra == 'docs'", specifier = ">=9,<10" }, { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.10,<3" },