Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,12 +332,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
mismatched keys are recorded as failed ledger statements and warned. Result
metadata reports the live tuned values for shuffle partitions and adaptive
execution.
- Tuning validation now rejects tuning requests the platform cannot execute,
and platforms without tuning coverage error on unsupported types like
covered platforms. The registry corrects several platforms' coverage
(Postgres family, Doris, SingleStore, Databend, Starburst, MotherDuck,
DataFusion, and fabric-warehouse, which now has its own entry instead of
sharing Databricks'), and the sample and fallback configuration builders
only enable tuning the target platform supports. Layout tuning on
BigQuery, Redshift, Trino and Synapse now errors until their execution
support lands.
- Tuned load-only runs (`--phases generate,load`) now go through the same
apply → load → reconcile → metadata → ledger/receipt → export lifecycle as
full runs, so a load-only bundle never carries a tuned mode without
`platform.tuning`. Adapters without the full lifecycle refuse tuned load-only
runs with an actionable error instead.

- Result metadata records genuine engine versions (`engine_version`, `engine_version_source: sql_query`) for DuckDB, SQLite and DataFusion queried directly from the active connection rather than package versions.

- ClickHouse tuning introspection bounds table queries in SQL by ledger tables and measures snapshot truncation directly on raw query results.
Expand Down Expand Up @@ -690,6 +698,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
how to choose the engine at run time; pass `--execution-engine` (for example
`--execution-engine streaming`). Tuning files that already set
`engine_affinity` keep working and record `resolution` `tuning_profile`.
- Tuning validation now fails when a configuration asks for tuning that the
platform does not apply during a run. Earlier releases accepted these
configurations and ran without the tuning. Run `benchbox tuning validate
--platform <platform>` on your saved tuning files before you upgrade; each
error names the tuning type and why it is not applied. Remove the setting
or use a platform that applies it. Platforms that apply tuning in part are
checked type by type. The new errors are:
- `unique_constraints` and `check_constraints` fail on every platform with
registry coverage, because no benchmark schema carries those clauses. The
packaged DuckDB and Snowflake templates and the matching
`examples/tunings` files now disable both.
- Layout tuning fails on BigQuery (`partitioning`, `clustering`), Redshift
(`partitioning`, `distribution`), Trino and
Azure Synapse (all four layout types). These adapters build the clauses
for dry-run previews but do not apply them when tables are created. Other
platforms that only preview layout tuning fail the same way (for example
the Postgres family, Doris, Firebolt, QuestDB, SingleStore and
TimescaleDB).
- Redshift `sorting` fails while `sorted_ingestion_mode` is `off`, which is
the default, because the adapter sorts tables only when sorted ingestion
is on. Set `sorted_ingestion_mode` to `auto` or `force` to keep the
request.
- `partitioning` fails on Databricks, and so on the `spark` and `delta`
platform names that resolve to it. The adapter builds the clause for
previews only and never adds it when tables are created. Fabric Warehouse
no longer shares the Databricks rules and also rejects `partitioning`.
- `primary_keys` and `foreign_keys` fail where the adapter removes that
constraint from table definitions: primary keys on Databend, Doris,
Firebolt, QuestDB, Trino and Presto; foreign keys on ClickHouse, Databend,
Doris, Firebolt, QuestDB, SingleStore and StarRocks. Constraints that
DataFusion and MotherDuck do not create fail the same way.
- Platforms without registry coverage now report unsupported tuning types as
errors where earlier releases reported warnings, and default tuning
configurations on platforms that drop tuning report errors where they
reported warnings.

### Added

Expand Down
8 changes: 0 additions & 8 deletions benchbox/cli/commands/tuning_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@
from benchbox.core.platform_manifest import get_platform_aliases
from benchbox.core.tuning.capability_registry import (
PLATFORM_ALIASES,
get_capability,
known_registry_platforms,
resolve_platform_key,
)
Expand Down Expand Up @@ -343,13 +342,6 @@ def _validate_sql_config(config_file: str, platform: str) -> None:

errors, warnings = config.validate_for_platform_detailed(platform_key)
enabled = sorted(config.get_enabled_tuning_types(), key=lambda tuning_type: tuning_type.value)
for tuning_type in enabled:
capability = get_capability(platform_key, tuning_type)
if capability is not None and capability.rendered_via == "none":
warnings.append(
f"Tuning type '{tuning_type.value}' is accepted by platform '{platform_key}' "
f"but not rendered yet ({capability.mechanism_id})"
)

for message in errors:
console.print(f"[red]Error: {message}[/red]")
Expand Down
11 changes: 11 additions & 0 deletions benchbox/cli/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -647,6 +647,17 @@ def create_sample_unified_tuning_config(self, path: Optional[Path] = None, platf
TuningColumn(name="linenumber", type="INTEGER", order=2),
]

from benchbox.core.tuning.capability_registry import get_capability, unapplied_reason

platform_known = TuningType.is_known_platform(platform)
for tuning_type in list(sample_config.get_enabled_tuning_types()):
capability = get_capability(platform, tuning_type)
if capability is not None:
if unapplied_reason(capability, sample_config.platform_optimizations.sorted_ingestion_mode):
sample_config.disable_tuning_type(tuning_type)
elif platform_known and not tuning_type.is_compatible_with_platform(platform):
sample_config.disable_tuning_type(tuning_type)

self.save_unified_tuning_config(sample_config, path)
console.print(
f"[green]✅ Sample unified tuning configuration created for platform '{platform}' at {path}[/green]"
Expand Down
10 changes: 9 additions & 1 deletion benchbox/cli/tuning_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,16 @@


def build_default_unified_config(platform: str | None) -> UnifiedTuningConfiguration:
from benchbox.core.tuning.capability_registry import get_capability, is_not_rendered_at_execution

config = UnifiedTuningConfiguration()
if platform and resolve_platform_key(str(platform).split(":", 1)[0]) == "duckdb":
if platform:
platform_key = resolve_platform_key(str(platform))
for tuning_type in config.get_enabled_tuning_types():
capability = get_capability(platform_key, tuning_type)
if capability is not None and is_not_rendered_at_execution(capability):
config.disable_tuning_type(tuning_type)
if platform and resolve_platform_key(str(platform)) == "duckdb":
config.check_constraints.enabled = False
return config

Expand Down
Loading
Loading