Skip to content

fix(clickhouse): support env:VAR / vault: spec syntax for credentials - #57

Merged
kazmosahebi merged 4 commits into
mainfrom
feat/clickhouse-env-credentials
Jun 22, 2026
Merged

kazmosahebi merged 4 commits into
mainfrom
feat/clickhouse-env-credentials

Conversation

@kazmosahebi

Copy link
Copy Markdown
Contributor

Summary

  • New src/config/credentials.rs: resolve_clickhouse_credentials(&mut ClickHouseConfig) resolves env:VAR / vault:path:key / literal specs on username and password via hyperi_rustlib::credential::resolve.
  • Wired into run_service's async move block — runs once at startup, before the From<&loader::ClickHouseConfig> bridge to the internal client config.
  • Adds "credential" to the existing hyperi-rustlib feature list (does not add "secrets" — vault paths return VaultUnsupported).
  • Updates config.example.yaml with inline comments documenting the spec syntax and an env:CLICKHOUSE_USERNAME/env:CLICKHOUSE_PASSWORD example block.

Closes #56.

Why

Today, if a deployment sets CLICKHOUSE_PASSWORD=foo in compose .env, the ClickHouse server gets configured with that password but the loader does not — its flat-env layer only reads DFE_LOADER_* prefixed vars, so CLICKHOUSE_PASSWORD is silently ignored and the loader falls back to the hard-coded default user with empty password. Auth then fails for any non-default password setup. This PR lets loader.yaml reference env vars directly (password: env:CLICKHOUSE_PASSWORD), matching the pattern dfe-fetcher already uses for AWS/GCP/M365/Azure credentials.

Field types — no breaking changes

The YAML-deser ClickHouseConfig at src/config/loader.rs:120-130 already has password: SensitiveString. The internal client struct at src/clickhouse/config.rs keeps password: String. The From impl at loader.rs:146-167 bridges them; resolution happens on the YAML struct before that bridge runs.

Dependencies

⚠️ End-to-end blocker

This feature is structurally complete and library-level tested, but cannot resolve credentials in production until hyperi-io/hyperi-rustlib#41 is fixed. That rustlib bug (SensitiveString::serialize returning the literal ***REDACTED*** string) destroys the YAML password value during this repo's apply_figment_env serde round-trip — the resolver then sees ***REDACTED*** instead of the original spec.

Diagnostic: temporarily swapping password: SensitiveString → password: String confirms make dev runs end-to-end and the credential resolver works as designed. Reverted before commit; see comment on #56 for the full trace.

Suggested merge order: rustlib#41 (SensitiveString fix) → rustlib#42 (credential extraction, ideally same release) → cut rustlib release → dfe-fetcher#26 (rustlib version bump + credential adoption) → this PR (rustlib version bump + bin-crate path fix already on this branch).

Commits

  • ef4c29b — Phase 3 implementation (resolver + module wiring + example config + Cargo.toml feature)
  • e64515e — fix: bin-crate path (crate::config::credentials → dfe_loader::config::credentials) — main.rs is a separate binary crate, so crate:: does not reach the lib.

Test plan

  • cargo test --lib config::credentials → 4/4 (literal pass-through, env resolved, missing-env clear error, vault-without-feature VaultUnsupported)
  • cargo test --lib → 1183/1183 pass
  • cargo clippy --lib -- -D warnings clean
  • make dev (dfe-docker) builds the loader container against this branch + local rustlib path override; container starts cleanly with the diagnostic-only SensitiveString → String patch applied (proves the resolver path works)
  • CI green (requires rustlib release containing chore(deps): Pin hyperi-io/hyperi-ci action to f81f531 #41 + chore(deps): Update hyperi-io/hyperi-ci digest to a1112e2 #42)
  • End-to-end auth-success on real ClickHouse (requires rustlib#41 fix)

kazmosahebi added 3 commits May 27, 2026 12:41
Resolves clickhouse.username and clickhouse.password as credential
specs at startup via hyperi_rustlib::credential::resolve. The YAML
field accepts env:VAR_NAME (read from environment, hard error if
unset), vault:path:key (requires the secrets rustlib feature; off
by default, so vault: returns VaultUnsupported), or a literal value.

Adds "credential" to the hyperi-rustlib feature list. Updates
config.example.yaml to document the new syntax.

Closes #56. Depends on hyperi-io/hyperi-rustlib#40.
`main.rs` is a separate binary crate whose `crate::` root has no `config`
module — only the library crate does. The Phase 3 commit's
`crate::config::credentials::...` therefore failed to compile when the
binary was built (the lib-only test runs missed this because they don't
touch main.rs).

Use `dfe_loader::config::credentials::...` instead, matching the existing
`use dfe_loader::config::{Config, ...}` pattern elsewhere in main.rs.
@kazmosahebi
kazmosahebi force-pushed the feat/clickhouse-env-credentials branch from 5bc0da9 to f68eea4 Compare May 27, 2026 02:42
@kazmosahebi kazmosahebi changed the title feat(clickhouse): support env:VAR / vault: spec syntax for credentials fix(clickhouse): support env:VAR / vault: spec syntax for credentials May 27, 2026
@kazmosahebi
kazmosahebi merged commit f79aa74 into main Jun 22, 2026
1 check passed
@kazmosahebi
kazmosahebi deleted the feat/clickhouse-env-credentials branch June 22, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support env:VAR spec syntax for ClickHouse credentials

1 participant