diff --git a/docs/design/parquet-v3-page-level-constant-column.md b/docs/design/parquet-v3-page-level-constant-column.md new file mode 100644 index 000000000000..80bb8c2de78d --- /dev/null +++ b/docs/design/parquet-v3-page-level-constant-column.md @@ -0,0 +1,110 @@ +# Parquet v3: per-subgroup (page-level) constant-column detection + +## Motivation + +The current constant-column optimization (`detectConstantColumn`) fires only when a whole +Parquet **column chunk** is single-valued (footer `min == max`, no nulls). Data that is sorted or +clustered on a column is often constant over long **runs of pages** without the whole row group +being constant, so chunk-level detection misses it. + +Parquet's **Column Index** stores per-page `min_values` / `max_values` / `null_pages` / +`null_counts`. The v3 reader already loads and parses it (`applyColumnIndex`) for predicate +push-down, so per-page "is this page constant?" is available at zero extra I/O. This lets us mark a +column constant for the row range of an individual **row subgroup** (the unit that becomes one +output `Chunk`) and materialize it as a `ColumnConst`, skipping the covered pages' reads and decode. + +## Tiers (both kept) + +| Tier | Source | Always present? | Exact flag? | Granularity | +|------|--------|-----------------|-------------|-------------| +| 1 (existing) | footer `ColumnMetaData.statistics` | yes | yes (`is_min_value_exact`) | whole chunk | +| 2 (this doc) | Column Index (per page) | no (optional) | no | per subgroup | + +Tier 1 stays the always-on baseline and the **only** safe detector for `BYTE_ARRAY` / +`FIXED_LEN_BYTE_ARRAY` (Column Index has no per-page exactness flag; 16-byte truncation can make two +distinct strings compare equal). Tier 2 is opportunistic: only for fixed-width numeric/date/time, +only when the Column Index is already loaded, only for chunks tier 1 did not already mark constant. + +## Key decision: do NOT change subgroup sizing + +Aligning subgroups to page boundaries would fragment the block stream into many tiny chunks +(pages are far smaller than a row group), and a subgroup carries all columns so its size is bounded +by the non-constant columns anyway. Instead, keep subgroup boundaries exactly as today and do +**per-subgroup, per-column** detection: a column is constant for a subgroup iff every Column-Index +page overlapping the subgroup's row range is constant with the *same* value (or every such page is +`null_pages`). Chunk count is unchanged; we just catch subgroups that sit inside a constant run. + +## Phases + +- **Phase 0** — retain per-page constant info. `applyColumnIndex` currently discards the parsed + `parq::ColumnIndex`. Keep a compact per-page summary on `ColumnChunk` (value + `is_const` + + `all_null`), plus the page→`first_row_index` map already in the Offset Index. Populate only for + eligible types. +- **Phase 1** — `detectConstantSubchunk(column, column_info, [start_row, end_row))`: scan the pages + overlapping the range; return constant + value when all are `is_const` and share one value; + all-null when all are `null_pages`. +- **Phase 2** — call it in `intersectColumnIndexResultsAndInitSubgroups` for each subgroup / + primitive column that tier 1 didn't already mark constant; set `subchunk.is_constant` / + `is_all_null` / `constant_value` (the same fields `decodePrimitiveColumn` propagates). + `formOutputColumn` needs no change (already emits `ColumnConst`). +- **Phase 3** — skip work for constant subchunks: `decodePrimitiveColumn` skips decode on + `subchunk.is_constant`; `determinePagesToPrefetch` skips fetching a page only when it is constant + in **every** subgroup that overlaps it. +- **Phase 4** — stateless tests (page-run constant, all-null-per-page, byte-array negative/truncation + guard, `GROUP BY` correctness), with a new `ParquetConstantColumnSubchunks` ProfileEvent to prove + tier 2 fired. +- **Phase 5** — ProfileEvents comparison on clustered data: expect further `S3GetObject` / + `ParquetFetchWaitTimeMicroseconds` drops with chunk count unchanged. + +## Optional force-load + +By default tier 2 only uses the Column Index when it is already loaded (columns with a predicate +push-down). `input_format_parquet_use_column_index_for_constant_columns` (default off) extends it: +the Column Index (+ Offset Index) is force-loaded for eligible read columns that have no predicate, +so tier 2 can also fire on them. Cost is a small extra read of the (tiny, tail-contiguous, coalesced) +index; worthwhile mainly for sorted / low-cardinality columns. `applyColumnIndex` records per-page +constant info but skips predicate pruning when the column has no condition. A future `auto` mode +could gate this on footer signals (row-group `sorting_columns`, low compressed-bytes-per-value, +dictionary encoding stats) instead of an all-or-nothing switch. + +## Cast safety (future-proofing) + +The optimization only fires when the stats decoder needs no value-transforming conversion +(`SchemaConverter` sets `allow_stats` accordingly), so today `input_type == output_type` for every +constant column and no cast is applied. To keep the constant path correct if `allow_stats` is ever +generalized to allow transforming casts, `formOutputColumn` builds the single-value constant in +`input_type` and runs the same `castColumn` the per-row decode uses when `needs_cast` is set - a +no-op today, O(1) on the `ColumnConst`, and it preserves const-ness. The all-null constant is +synthesized directly in the output domain (Null / output default), so it is not cast. The mixed fill +already goes through the normal `formOutputColumn` cast, so it is future-safe too. This does not +touch the `allow_stats` decision itself. + +## Guardrails + +- All types, including `BYTE_ARRAY` / `FIXED_LEN_BYTE_ARRAY` strings. No truncation guard is needed + for the `min == max` case: statistics/Column-Index bounds are always valid + (`min <= every value <= max`) and truncation only widens them, so `min == max` proves a single + exact value (a truncated or multi-valued page yields `min < max`). This holds at both the chunk + level (tier 1) and per page (tier 2), so neither needs the `is_*_value_exact` flag. +- Gate on the existing `input_format_parquet_use_constant_column_optimization` setting; the + force-load above is additionally gated by + `input_format_parquet_use_column_index_for_constant_columns`. +- Partial-page subgroup boundaries are fine: a partial overlap of a constant page still yields that + value, as long as every overlapping page is constant with the shared value. + +## Mixed-topology fill (Approach B) — implemented, default off + +`input_format_parquet_fill_constant_pages` handles constant runs *shorter* than / straddling a +subgroup: `fillConstantPagesAndDecodeRest` fills single-value pages from the Column Index and decodes +only the varying ones, and `determinePagesToPrefetch` skips prefetching the filled pages (a page +shared with a subgroup that decodes it normally is still fetched — `willFillConstantPages` is +deterministic so both paths agree). The fill walks the rows that pass the filter (`row_subgroup.filter`), so it works under PREWHERE / +row-level filters and page-pruning predicates too - it produces exactly `rows_pass` values for any +filter, and because the decision no longer depends on `rows_pass` it is identical at prefetch time +and decode time (so the prefetch-skip can never drop a page the decode needs). All-null pages are +filled (nulls via the compact values + null map + `expand` path, or the output default under +`null_as_default`); a subgroup with an all-null page falls back only when the output can represent +neither null nor a default. The one remaining gate is `needs_cast`: the fill writes the Column Index +value into the `decoded_type` column, valid only when no post-decode cast applies (resolving whether +`decodeField` yields the decoded or the output value domain is build-gated). Experimental, off by +default; needs a build + correctness tests before it can be trusted. diff --git a/docs/en/engines/database-engines/datalake.md b/docs/en/engines/database-engines/datalake.md index b37fc38f790d..45ba8f070f37 100644 --- a/docs/en/engines/database-engines/datalake.md +++ b/docs/en/engines/database-engines/datalake.md @@ -59,6 +59,7 @@ The following settings are supported: | `region` | AWS region for the service (e.g., `us-east-1`) | | `dlf_access_key_id` | Access key ID for DLF access | | `dlf_access_key_secret` | Access key Secret for DLF access | +| `namespaces` | Comma-separated list of namespaces, implemented for catalog types: `rest`, `glue` and `unity` | ## Examples {#examples} @@ -81,4 +82,30 @@ SETTINGS onelake_client_secret = client_secret; SHOW TABLES IN database_name; SELECT count() from database_name.table_name; -``` \ No newline at end of file +``` + +## Namespace filter {#namespace} + +By default, ClickHouse reads tables from all namespaces available in the catalog. You can limit this behavior using the `namespaces` database setting. The value should be a comma‑separated list of namespaces that are allowed to be read. + +Supported catalog types are `rest`, `glue` and `unity`. + +For example, if the catalog contains three namespaces - `dev`, `stage`, and `prod` - and you want to read data only from dev and stage, set: +``` +namespaces='dev,stage' +``` + +### Nested namespaces {#namespace-nested} + +The Iceberg (`rest`) catalog supports nested namespaces. The `namespaces` filter accepts the following patterns: + +- `namespace` - includes tables from the specified namespace, but not from its nested namespaces. +- `namespace.nested` - includes tables from the nested namespace, but not from the parent. +- `namespace.*` - includes tables from all nested namespaces, but not from the parent. + +If you need to include both a namespace and its nested namespaces, specify both explicitly. For example: +``` +namespaces='namespace,namespace.*' +``` + +The default value is '*', which means all namespaces are included. diff --git a/programs/local/LocalServer.cpp b/programs/local/LocalServer.cpp index b5b2c9ecb065..bd9f5454e3a2 100644 --- a/programs/local/LocalServer.cpp +++ b/programs/local/LocalServer.cpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -161,6 +162,9 @@ namespace ServerSetting extern const ServerSettingsString mark_cache_policy; extern const ServerSettingsUInt64 mark_cache_size; extern const ServerSettingsDouble mark_cache_size_ratio; + extern const ServerSettingsString object_storage_identity_cache_policy; + extern const ServerSettingsUInt64 object_storage_identity_cache_size; + extern const ServerSettingsDouble object_storage_identity_cache_size_ratio; extern const ServerSettingsString unique_key_bitmap_cache_policy; extern const ServerSettingsUInt64 unique_key_bitmap_cache_size_bytes; extern const ServerSettingsDouble unique_key_bitmap_cache_size_ratio; @@ -224,6 +228,7 @@ namespace ServerSetting extern const ServerSettingsUInt64 max_keep_alive_requests; extern const ServerSettingsBool asynchronous_metrics_enable_heavy_metrics; extern const ServerSettingsUInt32 asynchronous_heavy_metrics_update_period_s; + extern const ServerSettingsBool http_latency_aware_host_selection; } namespace ErrorCodes @@ -419,6 +424,8 @@ void LocalServer::initialize(Poco::Util::Application & self) server_settings[ServerSetting::max_format_parsing_thread_pool_size], server_settings[ServerSetting::max_format_parsing_thread_pool_free_size], server_settings[ServerSetting::format_parsing_thread_pool_queue_size]); + + HostResolver::setLatencyAwareSelection(server_settings[ServerSetting::http_latency_aware_host_selection]); } @@ -1422,6 +1429,11 @@ void LocalServer::processConfig() } global_context->setMarkCache(mark_cache_policy, mark_cache_size, mark_cache_size_ratio); + global_context->setObjectStorageIdentityCache( + server_settings[ServerSetting::object_storage_identity_cache_policy], + server_settings[ServerSetting::object_storage_identity_cache_size], + server_settings[ServerSetting::object_storage_identity_cache_size_ratio]); + /// UNIQUE KEY delete-bitmap cache. Zero size disables. String unique_key_bitmap_cache_policy_name = server_settings[ServerSetting::unique_key_bitmap_cache_policy]; size_t unique_key_bitmap_cache_size = server_settings[ServerSetting::unique_key_bitmap_cache_size_bytes]; diff --git a/programs/server/Server.cpp b/programs/server/Server.cpp index 9260a3c9d2dc..2f3f7a5c4471 100644 --- a/programs/server/Server.cpp +++ b/programs/server/Server.cpp @@ -60,6 +60,7 @@ #include #include #include +#include #include #include #include @@ -258,6 +259,7 @@ namespace ServerSetting extern const ServerSettingsUInt64 http_connections_warn_limit; extern const ServerSettingsUInt64 http_connections_rcvbuf; extern const ServerSettingsUInt64 http_connections_sndbuf; + extern const ServerSettingsBool http_latency_aware_host_selection; extern const ServerSettingsString index_mark_cache_policy; extern const ServerSettingsUInt64 index_mark_cache_size; extern const ServerSettingsDouble index_mark_cache_size_ratio; @@ -298,6 +300,9 @@ namespace ServerSetting extern const ServerSettingsString mark_cache_policy; extern const ServerSettingsUInt64 mark_cache_size; extern const ServerSettingsDouble mark_cache_size_ratio; + extern const ServerSettingsString object_storage_identity_cache_policy; + extern const ServerSettingsUInt64 object_storage_identity_cache_size; + extern const ServerSettingsDouble object_storage_identity_cache_size_ratio; extern const ServerSettingsString unique_key_index_cache_policy; extern const ServerSettingsUInt64 unique_key_index_cache_size_bytes; extern const ServerSettingsDouble unique_key_index_cache_size_ratio; @@ -2150,6 +2155,11 @@ try } global_context->setMarkCache(mark_cache_policy, mark_cache_size, mark_cache_size_ratio); + global_context->setObjectStorageIdentityCache( + server_settings[ServerSetting::object_storage_identity_cache_policy], + server_settings[ServerSetting::object_storage_identity_cache_size], + server_settings[ServerSetting::object_storage_identity_cache_size_ratio]); + String unique_key_index_cache_policy_name = server_settings[ServerSetting::unique_key_index_cache_policy]; size_t unique_key_index_cache_size = server_settings[ServerSetting::unique_key_index_cache_size_bytes]; double unique_key_index_cache_size_ratio = server_settings[ServerSetting::unique_key_index_cache_size_ratio]; @@ -2749,6 +2759,8 @@ try new_server_settings[ServerSetting::http_connections_sndbuf], }); + HostResolver::setLatencyAwareSelection(new_server_settings[ServerSetting::http_latency_aware_host_selection]); + DNSResolver::instance().setFilterSettings(new_server_settings[ServerSetting::dns_allow_resolve_names_to_ipv4], new_server_settings[ServerSetting::dns_allow_resolve_names_to_ipv6]); if (global_context->isServerCompletelyStarted()) diff --git a/src/Access/Common/AccessType.h b/src/Access/Common/AccessType.h index 38766a2ed170..245ec5b1d677 100644 --- a/src/Access/Common/AccessType.h +++ b/src/Access/Common/AccessType.h @@ -318,6 +318,7 @@ enum class AccessType : uint8_t M(SYSTEM_DROP_ICEBERG_METADATA_CACHE, "SYSTEM CLEAR ICEBERG_METADATA_CACHE, SYSTEM DROP ICEBERG_METADATA_CACHE", GLOBAL, SYSTEM_DROP_CACHE) \ M(SYSTEM_DROP_AVRO_SCHEMA_CACHE, "SYSTEM CLEAR AVRO SCHEMA CACHE, SYSTEM DROP AVRO SCHEMA CACHE, DROP AVRO SCHEMA CACHE", GLOBAL, SYSTEM_DROP_CACHE) \ M(SYSTEM_DROP_PARQUET_METADATA_CACHE, "SYSTEM DROP PARQUET_METADATA_CACHE", GLOBAL, SYSTEM_DROP_CACHE) \ + M(SYSTEM_DROP_OBJECT_STORAGE_IDENTITY_CACHE, "SYSTEM CLEAR OBJECT STORAGE IDENTITY CACHE, SYSTEM DROP OBJECT STORAGE IDENTITY CACHE", GLOBAL, SYSTEM_DROP_CACHE) \ M(SYSTEM_PREWARM_PRIMARY_INDEX_CACHE, "SYSTEM PREWARM PRIMARY INDEX, PREWARM PRIMARY INDEX CACHE, PREWARM PRIMARY INDEX", GLOBAL, SYSTEM_DROP_CACHE) \ M(SYSTEM_DROP_PRIMARY_INDEX_CACHE, "SYSTEM CLEAR PRIMARY INDEX CACHE, SYSTEM DROP PRIMARY INDEX, DROP PRIMARY INDEX CACHE, DROP PRIMARY INDEX", GLOBAL, SYSTEM_DROP_CACHE) \ M(SYSTEM_DROP_UNCOMPRESSED_CACHE, "SYSTEM CLEAR UNCOMPRESSED CACHE, SYSTEM DROP UNCOMPRESSED, DROP UNCOMPRESSED CACHE, DROP UNCOMPRESSED", GLOBAL, SYSTEM_DROP_CACHE) \ diff --git a/src/Common/CurrentMetrics.cpp b/src/Common/CurrentMetrics.cpp index cdb399e249ce..6d5efded9634 100644 --- a/src/Common/CurrentMetrics.cpp +++ b/src/Common/CurrentMetrics.cpp @@ -337,6 +337,8 @@ M(IcebergMetadataFilesCacheFiles, "Number of cached files in the Iceberg metadata cache") \ M(ParquetMetadataCacheBytes, "Size of the Parquet metadata cache in bytes") \ M(ParquetMetadataCacheFiles, "Number of cached files in the Parquet metadata cache") \ + M(ObjectStorageIdentityCacheBytes, "Size of the object-storage identity cache in bytes") \ + M(ObjectStorageIdentityCacheCells, "Number of entries in the object-storage identity cache") \ M(AvroSchemaCacheBytes, "Size of the Avro schema cache in bytes") \ M(AvroSchemaCacheCells, "Number of cached Avro schemas, including both registered and fetched schemas.") \ M(AvroSchemaRegistryCacheBytes, "Size of the Avro schema registry cache in bytes") \ diff --git a/src/Common/ErrorCodes.cpp b/src/Common/ErrorCodes.cpp index 07ec3fff851f..ad3113b17b1f 100644 --- a/src/Common/ErrorCodes.cpp +++ b/src/Common/ErrorCodes.cpp @@ -658,6 +658,7 @@ M(776, RESOURCE_LIMIT_EXCEEDED) \ M(777, MEMORY_RESERVATION_KILLED) \ M(778, MEMORY_RESERVATION_FAILED) \ + M(779, CATALOG_NAMESPACE_DISABLED) \ \ M(900, DISTRIBUTED_CACHE_ERROR) \ M(901, CANNOT_USE_DISTRIBUTED_CACHE) \ diff --git a/src/Common/HTTPConnectionPool.cpp b/src/Common/HTTPConnectionPool.cpp index 999003858b4c..da9f5259cc5d 100644 --- a/src/Common/HTTPConnectionPool.cpp +++ b/src/Common/HTTPConnectionPool.cpp @@ -1,6 +1,9 @@ #include #include +#include +#include + #include #include #include @@ -714,8 +717,81 @@ class EndpointConnectionPool : public std::enable_shared_from_this candidates; + candidates.reserve(stored_connections.size()); + while (!stored_connections.empty()) + { + candidates.push_back(stored_connections.top()); + stored_connections.pop(); + } + + auto resolver = HostResolversPool::instance().getResolver(host); + size_t best = 0; + double best_latency = std::numeric_limits::max(); + for (size_t i = 0; i < candidates.size(); ++i) + { + double latency = std::numeric_limits::max(); + Poco::Net::IPAddress ip; + if (Poco::Net::IPAddress::tryParse(candidates[i]->getResolvedHost(), ip)) + { + const double ewma = resolver->getLatencyMs(ip); + /// Unmeasured (0) is treated as worst so measured-fast connections + /// win; unmeasured ones are still used once the fast ones are handed + /// out (and reporting keeps every served address's EWMA fresh). + if (ewma > 0) + latency = ewma; + } + if (latency < best_latency) + { + best_latency = latency; + best = i; + } + } + + /// Pool-composition bias: if even the fastest pooled connection's front-end is + /// materially slower than the fastest front-end we know of, prefer opening a + /// fresh connection (selectBest routes it to a fast front-end) over reusing a + /// slow-backend one. Same-region the TCP connect cost is ~1 ms, negligible next + /// to the tens-of-ms backend TTFB gap, so this trades a cheap connect for a big + /// per-request win and shifts the pool's composition toward fast front-ends + /// over time. Gated on the soft limit so it cannot grow the pool without bound. + bool open_fast_instead = false; + if (!group->isSoftLimitReached() + && best_latency != std::numeric_limits::max()) + { + const double fleet_min = resolver->getMinLatencyMs(); + if (fleet_min > 0 && best_latency > fleet_min * 1.3 && best_latency - fleet_min > 15.0) + open_fast_instead = true; + } + + if (open_fast_instead) + { + /// Return everything to the pool; a fresh fast connection is opened below. + for (auto & c : candidates) + stored_connections.push(c); + } + else + { + reused_connection = candidates[best]; + for (size_t i = 0; i < candidates.size(); ++i) + if (i != best) + stored_connections.push(candidates[i]); + } + } + else + { + reused_connection = stored_connections.top(); + stored_connections.pop(); + } } } diff --git a/src/Common/HostResolvePool.cpp b/src/Common/HostResolvePool.cpp index e4688812f443..1cdcc73f3128 100644 --- a/src/Common/HostResolvePool.cpp +++ b/src/Common/HostResolvePool.cpp @@ -7,6 +7,8 @@ #include #include +#include +#include #include @@ -31,6 +33,18 @@ namespace ErrorCodes extern const int DNS_ERROR; } +std::atomic HostResolver::latency_aware_selection{false}; + +void HostResolver::setLatencyAwareSelection(bool enabled) +{ + latency_aware_selection.store(enabled, std::memory_order_relaxed); +} + +bool HostResolver::latencyAwareSelectionEnabled() +{ + return latency_aware_selection.load(std::memory_order_relaxed); +} + HostResolverMetrics HostResolver::getMetrics() { return HostResolverMetrics{ @@ -159,6 +173,51 @@ void HostResolver::setSuccess(const Poco::Net::IPAddress & address) updateWeights(); } +void HostResolver::reportLatency(const Poco::Net::IPAddress & address, UInt64 microseconds) +{ + if (microseconds == 0) + return; + + std::lock_guard lock(mutex); + + auto it = find(address); + if (it == records.end()) + return; + + it->setLatency(microseconds); + + /// A fresh latency sample shifts the EWMA, which changes the latency-biased weights even + /// when the base (usage) weight is unchanged, so recompute when latency-aware selection is on. + if (latency_aware_selection.load(std::memory_order_relaxed)) + updateWeights(); +} + +double HostResolver::getLatencyMs(const Poco::Net::IPAddress & address) +{ + std::lock_guard lock(mutex); + + auto it = find(address); + if (it == records.end()) + return 0; + + return it->latency_ms_ewma; +} + +double HostResolver::getMinLatencyMs() +{ + std::lock_guard lock(mutex); + + double best = 0; + for (const auto & rec : records) + { + if (rec.failed || rec.latency_ms_ewma <= 0) + continue; + if (best == 0 || rec.latency_ms_ewma < best) + best = rec.latency_ms_ewma; + } + return best; +} + void HostResolver::setFail(const Poco::Net::IPAddress & address) { Poco::Timestamp now; @@ -292,11 +351,47 @@ size_t HostResolver::getTotalWeight() const void HostResolver::updateWeightsImpl() { + /// When latency-aware selection is on, compute the median latency (TTFB EWMA) across the + /// addresses that have a measurement, and bias each address's base weight by how its own + /// latency compares to that median. The bias is clamped and floored so that no address is + /// starved: even a slow one keeps a weight of at least 1 and is still probed, which keeps + /// its EWMA fresh and lets it recover if it speeds up again. + double median_latency_ms = 0; + const bool latency_aware = latency_aware_selection.load(std::memory_order_relaxed); + if (latency_aware) + { + std::vector measured; + measured.reserve(records.size()); + for (const auto & rec : records) + if (rec.latency_ms_ewma > 0) + measured.push_back(rec.latency_ms_ewma); + + if (!measured.empty()) + { + auto mid = measured.begin() + measured.size() / 2; + std::nth_element(measured.begin(), mid, measured.end()); + median_latency_ms = *mid; + } + } + size_t total_weight_next = 0; for (auto & rec: records) { - total_weight_next += rec.getWeight(); + size_t weight = rec.getWeight(); + + if (latency_aware && weight > 0 && median_latency_ms > 0 && rec.latency_ms_ewma > 0) + { + /// ratio > 1 for faster-than-median addresses, < 1 for slower ones. Squared to + /// concentrate new-connection opens on the fast front-ends (so the pool composition + /// shifts toward them), with a wide clamp and a floor of 1 so slow addresses are + /// still probed occasionally and can recover. + const double ratio = median_latency_ms / rec.latency_ms_ewma; + const double factor = std::clamp(ratio * ratio, 0.1, 16.0); + weight = std::max(1, static_cast(std::lround(static_cast(weight) * factor))); + } + + total_weight_next += weight; rec.weight_prefix_sum = total_weight_next; } } diff --git a/src/Common/HostResolvePool.h b/src/Common/HostResolvePool.h index b007eadd6c4f..449c68db9c62 100644 --- a/src/Common/HostResolvePool.h +++ b/src/Common/HostResolvePool.h @@ -9,6 +9,7 @@ #include +#include #include #include @@ -28,6 +29,15 @@ // Addresses are resolved through `DB::DNSResolver::instance()`. // Usually it does not happen more often than 3 times in `history_` period. // But also new resolve performed each `setFail()` call. +// - latency-aware selection (optional, off by default) +// When enabled via `setLatencyAwareSelection()`, each address keeps an EWMA of its +// observed request latency (time-to-first-byte of read requests, reported via +// `setLatency()`). `selectBest()` biases the weighted random choice towards addresses +// whose latency is below the median across the resolved set, while every address keeps a +// floor of weight so it is still probed occasionally (keeps its EWMA fresh and lets a +// recovered address climb back). TTFB is used rather than TCP-connect RTT because the +// per-address latency spread is dominated by a stable, steerable backend-path component +// that connect RTT does not capture (connect RTT reflects only network proximity). namespace DB { @@ -107,6 +117,24 @@ class HostResolver : public std::enable_shared_from_this void update(); void reset(); + /// Enable/disable latency-aware address selection process-wide (all resolvers). Off by default. + static void setLatencyAwareSelection(bool enabled); + static bool latencyAwareSelectionEnabled(); + + /// Report a measured request latency (microseconds, typically time-to-first-byte) for an + /// address. Feeds the per-address latency EWMA used by latency-aware selection. Cheap no-op + /// when the address is unknown; only recomputes weights when latency-aware selection is on. + void reportLatency(const Poco::Net::IPAddress & address, UInt64 microseconds); + + /// Current latency EWMA (milliseconds) for an address, or 0 if unknown/not measured. + /// Used to pick the best pooled connection at dispatch time under latency-aware selection. + double getLatencyMs(const Poco::Net::IPAddress & address); + + /// Smallest latency EWMA (milliseconds) across all currently-known, non-failed addresses + /// that have a measurement, or 0 if none. Used to decide whether reusing a slow pooled + /// connection is worse than opening a fresh one to the fastest known front-end. + double getMinLatencyMs(); + static HostResolverMetrics getMetrics(); protected: @@ -145,6 +173,10 @@ class HostResolver : public std::enable_shared_from_this Poco::Timestamp fail_time = 0; size_t consecutive_fail_count = 0; + /// EWMA of observed request latency (time-to-first-byte) in milliseconds. 0 means + /// "not measured yet". Only used when latency-aware selection is enabled. + double latency_ms_ewma = 0; + size_t weight_prefix_sum{}; bool operator <(const Record & r) const @@ -196,6 +228,19 @@ class HostResolver : public std::enable_shared_from_this consecutive_fail_count = 0; ++usage; } + + void setLatency(UInt64 microseconds) + { + if (microseconds == 0) + return; + + /// Alpha weights the newest sample; small enough to smooth per-request jitter (and the + /// per-object backend-fetch variance) but responsive enough to follow a front-end that + /// persistently gets slower/faster. + static constexpr double alpha = 0.2; + const double ms = static_cast(microseconds) / 1000.0; + latency_ms_ewma = latency_ms_ewma > 0 ? alpha * ms + (1 - alpha) * latency_ms_ewma : ms; + } }; using Records = std::vector; @@ -223,6 +268,9 @@ class HostResolver : public std::enable_shared_from_this Poco::Timestamp last_resolve_time TSA_GUARDED_BY(mutex) = Poco::Timestamp::TIMEVAL_MIN; Records records TSA_GUARDED_BY(mutex); + /// Process-wide toggle for latency-aware selection. Shared by all resolvers. + static std::atomic latency_aware_selection; + Poco::Logger * log = &Poco::Logger::get("ConnectionPool"); }; diff --git a/src/Common/ProfileEvents.cpp b/src/Common/ProfileEvents.cpp index 7f35ef869e92..61ed35fecbf5 100644 --- a/src/Common/ProfileEvents.cpp +++ b/src/Common/ProfileEvents.cpp @@ -754,6 +754,9 @@ The server successfully detected this situation and will download merged part fr M(S3CopyObject, "Number of S3 API CopyObject calls.", ValueType::Number) \ M(S3ListObjects, "Number of S3 API ListObjects calls.", ValueType::Number) \ M(S3HeadObject, "Number of S3 API HeadObject calls.", ValueType::Number) \ + M(S3GetObjectAttributes, "Number of S3 API GetObjectAttributes calls.", ValueType::Number) \ + M(ObjectStorageIdentityCacheHits, "Number of object-storage identity (size/etag/part-offsets) cache hits, avoiding a HEAD/GetObjectAttributes.", ValueType::Number) \ + M(ObjectStorageIdentityCacheMisses, "Number of object-storage identity cache misses that triggered a metadata request.", ValueType::Number) \ M(S3GetObjectTagging, "Number of S3 API GetObjectTagging calls.", ValueType::Number) \ M(S3CreateMultipartUpload, "Number of S3 API CreateMultipartUpload calls.", ValueType::Number) \ M(S3UploadPartCopy, "Number of S3 API UploadPartCopy calls.", ValueType::Number) \ @@ -767,6 +770,7 @@ The server successfully detected this situation and will download merged part fr M(DiskS3CopyObject, "Number of DiskS3 API CopyObject calls.", ValueType::Number) \ M(DiskS3ListObjects, "Number of DiskS3 API ListObjects calls.", ValueType::Number) \ M(DiskS3HeadObject, "Number of DiskS3 API HeadObject calls.", ValueType::Number) \ + M(DiskS3GetObjectAttributes, "Number of DiskS3 API GetObjectAttributes calls.", ValueType::Number) \ M(DiskS3GetObjectTagging, "Number of DiskS3 API GetObjectTagging calls.", ValueType::Number) \ M(DiskS3CreateMultipartUpload, "Number of DiskS3 API CreateMultipartUpload calls.", ValueType::Number) \ M(DiskS3UploadPartCopy, "Number of DiskS3 API UploadPartCopy calls.", ValueType::Number) \ @@ -1442,11 +1446,22 @@ The server successfully detected this situation and will download merged part fr \ M(ParquetReadRowGroups, "The total number of row groups read from parquet data", ValueType::Number) \ M(ParquetPrunedRowGroups, "The total number of row groups pruned from parquet data", ValueType::Number) \ + M(ParquetConstantColumnChunks, "The total number of parquet column chunks materialized from a single value in their min/max statistics, without reading their data pages", ValueType::Number) \ + M(ParquetConstantColumnSubchunks, "The total number of parquet column subchunks (per row subgroup) materialized from a single value in their per-page column-index statistics, without decoding their data pages", ValueType::Number) \ M(ParquetDecodingTasks, "Tasks issued by parquet reader", ValueType::Number) \ M(ParquetDecodingTaskBatches, "Task groups sent to a thread pool by parquet reader", ValueType::Number) \ M(ParquetPrefetcherReadRandomRead, "The total number of reads with ReadMode::RandomRead by DB::Parquet::Prefetcher", ValueType::Number) \ M(ParquetPrefetcherReadSeekAndRead, "The total number of reads with ReadMode::SeekAndRead by DB::Parquet::Prefetcher", ValueType::Number) \ M(ParquetPrefetcherReadEntireFile, "The total number of read with ReadMode::EntireFileIsInMemory by DB::Parquet::Prefetcher", ValueType::Number) \ + M(ParquetPrefetcherServedFromRetainedTail, "The number of ranges (e.g. Column/Offset Index) served from the retained footer tail by DB::Parquet::Prefetcher without issuing a read", ValueType::Number) \ + M(ParquetPrefetcherPartAlignedTasks, "The number of read tasks whose coalescing was constrained to a single S3 multipart-upload part boundary by DB::Parquet::Prefetcher", ValueType::Number) \ + M(ParquetPrefetcherAlignmentSkippedSmall, "The number of times DB::Parquet::Prefetcher skipped read alignment because the aligned segment would be smaller than the configured minimum (anti-fragmentation)", ValueType::Number) \ + M(ParquetPrefetcherHedgedReads, "The number of hedged (duplicate) reads issued by DB::Parquet::Prefetcher to cut read tail latency", ValueType::Number) \ + M(ParquetPrefetcherHedgedWins, "The number of hedged reads that produced the result used (beat or replaced the primary read) in DB::Parquet::Prefetcher", ValueType::Number) \ + M(ParquetPrefetcherSplitReadTasks, "The number of read tasks that DB::Parquet::Prefetcher split into per-part-boundary segments read in parallel (to avoid a boundary-straddling GET)", ValueType::Number) \ + M(ParquetPrefetcherSplitReadSegments, "The total number of parallel segment reads produced by splitting boundary-straddling read tasks in DB::Parquet::Prefetcher", ValueType::Number) \ + M(ParquetPrefetcherFooterSpeculativeParallel, "The number of parquet footers read via a speculative-parallel tail read (last chunk + the rest fired concurrently) by DB::Parquet::Prefetcher", ValueType::Number) \ + M(ParquetPrefetcherFillRatioLimitedTasks, "The number of times DB::Parquet::Prefetcher stopped extending a coalesced read task because bridging the next gap would make the task mostly unwanted filler bytes (read_min_fill_ratio guard)", ValueType::Number) \ M(ParquetRowsFilterExpression, "The total number of rows that were passed through filter", ValueType::Number) \ M(ParquetColumnsFilterExpression, "The total number of columns that were passed through filter", ValueType::Number) \ M(FilterTransformPassedRows, "Number of rows that passed the filter in the query", ValueType::Number) \ diff --git a/src/Common/threadPoolCallbackRunner.cpp b/src/Common/threadPoolCallbackRunner.cpp index b4f918147557..ff5af3e74b5e 100644 --- a/src/Common/threadPoolCallbackRunner.cpp +++ b/src/Common/threadPoolCallbackRunner.cpp @@ -61,7 +61,7 @@ void ThreadPoolCallbackRunnerFast::startMoreThreadsIfNeeded(size_t active_tasks_ } } -void ThreadPoolCallbackRunnerFast::operator()(std::function f) +void ThreadPoolCallbackRunnerFast::operator()(std::function f, bool front) { if (mode == Mode::Disabled) throw Exception(ErrorCodes::LOGICAL_ERROR, "Thread pool runner is not initialized"); @@ -70,7 +70,11 @@ void ThreadPoolCallbackRunnerFast::operator()(std::function f) { std::unique_lock lock(mutex); - queue.push_back(std::move(f)); + /// Latency-critical tasks jump ahead of already-queued (e.g. prefetch) work. + if (front) + queue.push_front(std::move(f)); + else + queue.push_back(std::move(f)); startMoreThreadsIfNeeded(active_tasks_, lock); } diff --git a/src/Common/threadPoolCallbackRunner.h b/src/Common/threadPoolCallbackRunner.h index c65b23cfecbc..c3ccb01e3bfc 100644 --- a/src/Common/threadPoolCallbackRunner.h +++ b/src/Common/threadPoolCallbackRunner.h @@ -315,7 +315,10 @@ class ThreadPoolCallbackRunnerFast void shutdown(); - void operator()(std::function f); + /// front=true enqueues at the head of the queue so the task is picked up before already-queued + /// tasks. Use for latency-critical work (e.g. a read a consumer is actively blocked on) that + /// should jump ahead of speculative/prefetch work. + void operator()(std::function f, bool front = false); void bulkSchedule(std::vector> fs); diff --git a/src/Core/FormatFactorySettings.h b/src/Core/FormatFactorySettings.h index 67396c955eac..0c29d5baf351 100644 --- a/src/Core/FormatFactorySettings.h +++ b/src/Core/FormatFactorySettings.h @@ -206,6 +206,42 @@ Skip pages using min/max values from column index. )", 0) \ DECLARE(Bool, input_format_parquet_use_offset_index, true, R"( Minor tweak to how pages are read from parquet file when no page filtering is used. +)", 0) \ + DECLARE(Bool, input_format_parquet_use_constant_column_optimization, true, R"( +When a Parquet column chunk provably holds a single value in every row (according to its min/max statistics), materialize that value directly instead of reading and decoding the column's data pages. +)", 0) \ + DECLARE(Bool, input_format_parquet_use_column_index_for_constant_columns, false, R"( +Load the Parquet Column Index for read columns that have no predicate of their own, so the constant-column optimization can also skip data pages that are single-valued over a row subgroup (not just over a whole column chunk). Costs a small extra read of the (tiny) Column Index; only worthwhile when columns are sorted or low-cardinality. Applies only when `input_format_parquet_use_constant_column_optimization` is enabled. +)", 0) \ + DECLARE(Bool, input_format_parquet_fill_constant_pages, false, R"( +Experimental. When a Parquet column is single-valued over some data pages but not the whole row subgroup, fill those pages' rows from the per-page Column Index statistics instead of reading and decoding them (mixed-topology subgroups). Extends the constant-column optimization below the subgroup granularity. Requires `input_format_parquet_use_constant_column_optimization`; disabled by default. +)", 0) \ + DECLARE(Bool, input_format_parquet_align_reads_to_multipart_boundaries, false, R"( +Experimental. Align coalesced Parquet read requests to the boundaries of the object's S3 multipart-upload parts, so a single read never straddles two parts (an AWS best practice). Requires the per-file multipart layout, learned via GetObjectAttributes and cached (see the object-storage identity cache); has no effect for single-part objects or stores that don't expose part info. Disabled by default. +)", 0) \ + DECLARE(UInt64, input_format_parquet_read_alignment_bytes, 0, R"( +Experimental. Align coalesced Parquet read requests to a fixed byte grid of this size, so no read straddles a multiple of it. Set to the writer's multipart part size (e.g. 10Mi for delta-rs, 64Mi for Spark/S3A) to keep reads within single parts without probing per-file layout. 0 disables. When `input_format_parquet_align_reads_to_multipart_boundaries` is enabled and the real per-file part layout is known, that takes precedence over this fixed grid. +)", 0) \ + DECLARE(UInt64, input_format_parquet_read_alignment_min_bytes, 1048576, R"( +Experimental. Anti-fragmentation guard for Parquet read alignment (`input_format_parquet_read_alignment_bytes` / `input_format_parquet_align_reads_to_multipart_boundaries`): do not cut a read at a boundary when the resulting aligned segment would be smaller than this many bytes; allow the straddle instead of emitting a tiny extra request. Default 1 MiB. +)", 0) \ + DECLARE(Bool, input_format_parquet_split_reads_across_part_boundaries, false, R"( +Experimental. When a single Parquet read would straddle a part boundary (S3 multipart-upload part, from `input_format_parquet_align_reads_to_multipart_boundaries`, or a fixed grid from `input_format_parquet_read_alignment_bytes`), split it into per-part segments fetched in parallel instead of one boundary-crossing request. Measured on same-region AWS S3: a boundary-straddling ranged GET pays roughly one extra round-trip mid-stream, so aligned parallel reads are up to ~2.5x faster. Also enables a speculative-parallel Parquet footer read (the last 2 MiB and the rest of the hinted tail fired concurrently, avoiding a sequential second round-trip when the footer overflows 2 MiB). Only helps latency-bound / critical-path reads; the per-request win is hidden when concurrency is already saturated. Disabled by default. +)", 0) \ + DECLARE(Float, input_format_parquet_read_min_fill_ratio, 0, R"( +Experimental. Anti-amplification guard for Parquet v3 read coalescing. Coalescing reads through gaps shorter than the seek threshold to save requests, but many such sub-threshold gaps can accumulate into a read task that is almost entirely unwanted filler - e.g. reading a tiny, scattered (RLE / near-constant) column drags in the neighbouring columns and turns a few KB of wanted data into hundreds of MB read. When set above 0, a coalesced task is never allowed to be less than this fraction "wanted" (wanted bytes / total span); extension stops instead of bridging a gap that would dilute it below the ratio. Gaps below a small absolute floor are always bridged, so dense reads (whose fill stays ~1) are unaffected. Range 0..1; 0 disables (pure gap-size coalescing). Trades more, smaller requests for far fewer bytes on sparse reads - beneficial when request concurrency can hide the extra round-trips. Disabled by default. +)", 0) \ + DECLARE(UInt64, input_format_parquet_hedged_read_threshold_ms, 0, R"( +Experimental. Tail-latency mitigation for the Parquet v3 reader on remote object storage: if a read a query is blocked on has not completed within this many milliseconds, issue a duplicate (hedged) request and use whichever returns first. Cuts the S3 GET p99 tail at the cost of a few extra requests. 0 disables. Only reads no larger than `input_format_parquet_hedged_read_max_bytes` are hedged, and at most `input_format_parquet_hedged_read_max_inflight` hedges run at once. +)", 0) \ + DECLARE(UInt64, input_format_parquet_hedged_read_max_bytes, 4194304, R"( +Experimental. Only hedge Parquet reads (see `input_format_parquet_hedged_read_threshold_ms`) no larger than this - hedging targets latency of small/critical reads, not throughput of large coalesced reads. 0 = no size limit. Default 4 MiB. +)", 0) \ + DECLARE(UInt64, input_format_parquet_hedged_read_max_inflight, 4, R"( +Experimental. Cap on concurrent hedged Parquet reads (see `input_format_parquet_hedged_read_threshold_ms`), so a slow region cannot double all traffic. Default 4. +)", 0) \ + DECLARE(Double, input_format_parquet_prefetch_bandwidth_hide_seconds, 0, R"( +Read back-pressure for the Parquet v3 reader. When greater than zero, stop prefetching more compressed data pages ahead of decoding once the in-flight compressed bytes exceed this many seconds' worth of the measured read throughput (i.e. once the storage link is kept busy). Prevents buffering compressed data far beyond what bandwidth can consume. 0 disables the back-pressure (compressed prefetch is then bounded only by its memory budget). )", 0) \ DECLARE(Bool, input_format_parquet_verify_checksums, true, R"( Verify page checksums when reading parquet files. diff --git a/src/Core/ServerSettings.cpp b/src/Core/ServerSettings.cpp index 1f6eb7e54653..d895f2385639 100644 --- a/src/Core/ServerSettings.cpp +++ b/src/Core/ServerSettings.cpp @@ -542,6 +542,9 @@ namespace )", 0) \ DECLARE(Double, mark_cache_size_ratio, DEFAULT_MARK_CACHE_SIZE_RATIO, R"(The size of the protected queue (in case of SLRU policy) in the mark cache relative to the cache's total size.)", 0) \ DECLARE(Double, mark_cache_prewarm_ratio, 0.95, R"(The ratio of total size of mark cache to fill during prewarm.)", 0) \ + DECLARE(String, object_storage_identity_cache_policy, "SLRU", R"(Object-storage identity cache policy name (SLRU or LRU).)", 0) \ + DECLARE(UInt64, object_storage_identity_cache_size, 67108864, R"(Maximum size of the object-storage identity cache in bytes. This cache stores per-object size, ETag and multipart part offsets so that opening an object does not require a fresh HEAD/GetObjectAttributes on every read. Set to 0 to disable.)", 0) \ + DECLARE(Double, object_storage_identity_cache_size_ratio, 0.5, R"(The size of the protected queue (in case of SLRU policy) in the object-storage identity cache relative to the cache's total size.)", 0) \ DECLARE(String, unique_key_index_cache_policy, "SLRU", R"(UNIQUE KEY index cache policy name (SLRU or LRU).)", 0) \ DECLARE(UInt64, unique_key_index_cache_size_bytes, 1_GiB, R"(Maximum size (bytes) of the in-process cache for UNIQUE KEY index (SST) blocks. Set to 0 to disable the cache.)", 0) \ DECLARE(Double, unique_key_index_cache_size_ratio, 0.5, R"(The size of the protected queue (in case of SLRU policy) in the UNIQUE KEY index cache relative to the cache's total size.)", 0) \ @@ -1091,6 +1094,7 @@ The policy on how to perform a scheduling of CPU slots specified by `concurrent_ DECLARE(UInt64, storage_connections_sndbuf, 0, R"(The size of the SO_SNDBUF option for storage connections (replication, distributed queries). If set to a value greater than 0, overrides the kernel TCP autotuning for the send buffer. 0 = kernel default (autotuning). Note: changing this setting back to 0 restores autotuning only for newly created connections; existing pooled connections retain fixed buffer sizes until they are recreated.)", 0) \ DECLARE(UInt64, http_connections_rcvbuf, 0, R"(The size of the SO_RCVBUF option for general HTTP connections. If set to a value greater than 0, overrides the kernel TCP autotuning for the receive buffer. 0 = kernel default (autotuning). Note: changing this setting back to 0 restores autotuning only for newly created connections; existing pooled connections retain fixed buffer sizes until they are recreated.)", 0) \ DECLARE(UInt64, http_connections_sndbuf, 0, R"(The size of the SO_SNDBUF option for general HTTP connections. If set to a value greater than 0, overrides the kernel TCP autotuning for the send buffer. 0 = kernel default (autotuning). Note: changing this setting back to 0 restores autotuning only for newly created connections; existing pooled connections retain fixed buffer sizes until they are recreated.)", 0) \ + DECLARE(Bool, http_latency_aware_host_selection, false, R"(When a host name resolves to several addresses (for example an object storage endpoint behind a rotating set of front-end IPs), bias new-connection address selection towards the addresses with the lowest observed request latency. Each address keeps an EWMA of its measured time-to-first-byte; faster-than-median addresses get proportionally more of the weighted-random choice, while every address keeps a minimal weight so it is still probed. This captures per-front-end differences in backend-path latency, which the connect round-trip time alone does not reflect. Off by default. Affects only newly opened connections, not pooled ones.)", 0) \ DECLARE(UInt64, global_profiler_real_time_period_ns, 10000000000, R"(Period for real clock timer of global profiler (in nanoseconds). Set 0 value to turn off the real clock global profiler. Recommended value is at least 10000000 (100 times a second) for single queries or 1000000000 (once a second) for cluster-wide profiling.)", 0) \ DECLARE(UInt64, global_profiler_cpu_time_period_ns, 10000000000, R"(Period for CPU clock timer of global profiler (in nanoseconds). Set 0 value to turn off the CPU clock global profiler. Recommended value is at least 10000000 (100 times a second) for single queries or 1000000000 (once a second) for cluster-wide profiling.)", 0) \ DECLARE(Bool, enable_azure_sdk_logging, false, R"(Enables logging from Azure sdk)", 0) \ diff --git a/src/Core/SettingsChangesHistory.cpp b/src/Core/SettingsChangesHistory.cpp index c2b62c209863..ce59a1847044 100644 --- a/src/Core/SettingsChangesHistory.cpp +++ b/src/Core/SettingsChangesHistory.cpp @@ -41,6 +41,18 @@ const VersionToSettingsChangesMap & getSettingsChangesHistory() /// Note: please check if the key already exists to prevent duplicate entries. addSettingsChanges(settings_changes_history, "26.6", { + {"input_format_parquet_use_constant_column_optimization", false, true, "New setting: when a Parquet column chunk provably holds a single value in every row (per its min/max statistics), materialize that value directly instead of reading and decoding the column's data pages (reader v3)."}, + {"input_format_parquet_prefetch_bandwidth_hide_seconds", 0, 0, "New setting: read back-pressure for the Parquet v3 reader; stop prefetching compressed data pages once in-flight compressed bytes exceed this many seconds of measured read throughput. 0 (the default and the pre-existing behavior) disables the back-pressure."}, + {"input_format_parquet_use_column_index_for_constant_columns", false, false, "New setting: load the Parquet Column Index for read columns without a predicate so the constant-column optimization can skip data pages that are single-valued over a row subgroup. Disabled by default (costs a small extra read)."}, + {"input_format_parquet_fill_constant_pages", false, false, "New experimental setting: fill single-valued Parquet data pages from per-page Column Index statistics instead of decoding them, even when only part of a row subgroup is constant. Disabled by default."}, + {"input_format_parquet_align_reads_to_multipart_boundaries", false, false, "New experimental setting: align coalesced Parquet read requests to S3 multipart-upload part boundaries so a single read never straddles two parts. Disabled by default."}, + {"input_format_parquet_read_alignment_bytes", 0, 0, "New experimental setting: align coalesced Parquet reads to a fixed byte grid (e.g. the writer's multipart part size) without probing per-file layout. 0 disables."}, + {"input_format_parquet_read_alignment_min_bytes", 1048576, 1048576, "New experimental setting: anti-fragmentation guard for Parquet read alignment - do not split a read at a boundary when the aligned segment would be smaller than this. Default 1 MiB."}, + {"input_format_parquet_split_reads_across_part_boundaries", false, false, "New experimental setting: split a boundary-straddling Parquet read into per-part segments fetched in parallel (a straddling S3 GET pays ~one extra round-trip), and enable the speculative-parallel footer read. Disabled by default."}, + {"input_format_parquet_hedged_read_threshold_ms", 0, 0, "New experimental setting: hedge a Parquet read (issue a duplicate, take the winner) if it exceeds this many ms, to cut the S3 GET tail latency. 0 disables."}, + {"input_format_parquet_read_min_fill_ratio", 0., 0., "New experimental setting: anti-amplification guard for Parquet v3 read coalescing - stop bridging gaps once a coalesced task would be less than this fraction wanted bytes, so a tiny scattered column no longer drags in its neighbours. 0 disables."}, + {"input_format_parquet_hedged_read_max_bytes", 4194304, 4194304, "New experimental setting: only hedge Parquet reads no larger than this many bytes. Default 4 MiB."}, + {"input_format_parquet_hedged_read_max_inflight", 4, 4, "New experimental setting: cap on concurrent hedged Parquet reads. Default 4."}, {"analyzer_compatibility_allow_non_aggregate_in_having", false, false, "New compatibility setting. When enabled, the new analyzer mimics the legacy `HAVING`-to-`WHERE` rewrite for non-aggregate AND-conjuncts instead of raising `NOT_AN_AGGREGATE`."}, {"reserve_memory", 0, 0, "New setting to reserve memory for specific workload before starting a query."}, {"output_format_image_width", 1024, 1024, "New setting controlling the width of the output image for image output formats such as PNG."}, diff --git a/src/Databases/DataLake/DatabaseDataLake.cpp b/src/Databases/DataLake/DatabaseDataLake.cpp index 12fbb051ba4a..488fcba7eed7 100644 --- a/src/Databases/DataLake/DatabaseDataLake.cpp +++ b/src/Databases/DataLake/DatabaseDataLake.cpp @@ -66,6 +66,7 @@ namespace DatabaseDataLakeSetting extern const DatabaseDataLakeSettingsString aws_access_key_id; extern const DatabaseDataLakeSettingsString aws_secret_access_key; extern const DatabaseDataLakeSettingsString region; + extern const DatabaseDataLakeSettingsString namespaces; extern const DatabaseDataLakeSettingsString aws_role_arn; extern const DatabaseDataLakeSettingsString aws_role_session_name; extern const DatabaseDataLakeSettingsString aws_external_id; @@ -104,6 +105,7 @@ namespace Setting namespace DataLakeStorageSetting { extern const DataLakeStorageSettingsString iceberg_metadata_file_path; + extern const DataLakeStorageSettingsString iceberg_metadata_table_uuid; extern const DataLakeStorageSettingsBool iceberg_use_version_hint; } @@ -178,6 +180,7 @@ void DatabaseDataLake::initialize() const .aws_access_key_id = settings[DatabaseDataLakeSetting::aws_access_key_id].value, .aws_secret_access_key = settings[DatabaseDataLakeSetting::aws_secret_access_key].value, .region = settings[DatabaseDataLakeSetting::region].value, + .namespaces = settings[DatabaseDataLakeSetting::namespaces].value, .aws_role_arn = settings[DatabaseDataLakeSetting::aws_role_arn].value, .aws_role_session_name = settings[DatabaseDataLakeSetting::aws_role_session_name].value, .aws_external_id = settings[DatabaseDataLakeSetting::aws_external_id].value, @@ -195,6 +198,7 @@ void DatabaseDataLake::initialize() const settings[DatabaseDataLakeSetting::auth_header], settings[DatabaseDataLakeSetting::oauth_server_uri].value, settings[DatabaseDataLakeSetting::oauth_server_use_request_body].value, + settings[DatabaseDataLakeSetting::namespaces].value, Context::getGlobalContextInstance()); break; } @@ -248,6 +252,7 @@ void DatabaseDataLake::initialize() const settings[DatabaseDataLakeSetting::warehouse].value, url, settings[DatabaseDataLakeSetting::catalog_credential].value, + settings[DatabaseDataLakeSetting::namespaces].value, Context::getGlobalContextInstance()); break; } @@ -633,6 +638,12 @@ StoragePtr DatabaseDataLake::tryGetTableImpl(const String & name, ContextPtr con (*storage_settings)[DB::DataLakeStorageSetting::iceberg_metadata_file_path] = metadata_location; } + /// Forward the Iceberg table-uuid the catalog already parsed from its LoadTable response (see + /// RestCatalog::setTableUUID). It lets the Iceberg metadata layer key its files cache on the + /// first (bootstrap) metadata read, avoiding a redundant re-read of the same metadata file. + if (auto catalog_table_uuid = table_metadata.getTableUUID(); catalog_table_uuid.has_value()) + (*storage_settings)[DB::DataLakeStorageSetting::iceberg_metadata_table_uuid] = *catalog_table_uuid; + const auto configuration = getConfiguration(storage_type, storage_settings); /// HACK: Hacky-hack to enable lazy load diff --git a/src/Databases/DataLake/DatabaseDataLakeSettings.cpp b/src/Databases/DataLake/DatabaseDataLakeSettings.cpp index 969b0769d13a..a982ea0a7663 100644 --- a/src/Databases/DataLake/DatabaseDataLakeSettings.cpp +++ b/src/Databases/DataLake/DatabaseDataLakeSettings.cpp @@ -48,6 +48,7 @@ namespace ErrorCodes DECLARE(String, dlf_access_key_id, "", "Access id of DLF token for Paimon REST Catalog", 0) \ DECLARE(String, dlf_access_key_secret, "", "Access secret of DLF token for Paimon REST Catalog", 0) \ DECLARE(Bool, force_add_bucket, false, "Add bucket name to the metadata path", 0) \ + DECLARE(String, namespaces, "*", "Comma-separated list of allowed namespaces", 0) \ #define LIST_OF_DATABASE_ICEBERG_SETTINGS(M, ALIAS) \ DATABASE_ICEBERG_RELATED_SETTINGS(M, ALIAS) \ diff --git a/src/Databases/DataLake/GlueCatalog.cpp b/src/Databases/DataLake/GlueCatalog.cpp index 53ac171c79ff..67878334bacd 100644 --- a/src/Databases/DataLake/GlueCatalog.cpp +++ b/src/Databases/DataLake/GlueCatalog.cpp @@ -54,11 +54,14 @@ #include #include +#include + namespace DB::ErrorCodes { extern const int BAD_ARGUMENTS; extern const int DATALAKE_DATABASE_ERROR; extern const int FAULT_INJECTED; + extern const int CATALOG_NAMESPACE_DISABLED; } namespace DB::FailPoints @@ -172,6 +175,8 @@ GlueCatalog::GlueCatalog( LOG_TRACE(log, "Creating AWS glue client with credentials empty {}, region '{}', endpoint '{}'", credentials.IsEmpty(), region, endpoint); } + boost::split(allowed_namespaces, settings.namespaces, boost::is_any_of(", "), boost::token_compress_on); + credentials_provider = DB::S3::getCredentialsProvider(poco_config, credentials, creds_config); glue_client = std::make_unique(credentials_provider, endpoint_provider, client_configuration); @@ -200,8 +205,9 @@ DataLake::ICatalog::Namespaces GlueCatalog::getDatabases(const std::string & pre for (const auto & db : dbs) { const auto & db_name = db.GetName(); - if (!db_name.starts_with(prefix)) + if (!isNamespaceAllowed(db_name) || !db_name.starts_with(prefix)) continue; + result.push_back(db_name); if (limit != 0 && result.size() >= limit) break; @@ -286,6 +292,9 @@ DB::Names GlueCatalog::getTables() const bool GlueCatalog::existsTable(const std::string & database_name, const std::string & table_name) const { + if (!isNamespaceAllowed(database_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, "Namespace {} is filtered by `namespaces` database parameter", database_name); + Aws::Glue::Model::GetTableRequest request; request.SetDatabaseName(database_name); request.SetName(table_name); @@ -299,6 +308,9 @@ bool GlueCatalog::tryGetTableMetadata( const std::string & table_name, TableMetadata & result) const { + if (!isNamespaceAllowed(database_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, "Namespace {} is filtered by `namespaces` database parameter", database_name); + Aws::Glue::Model::GetTableRequest request; request.SetDatabaseName(database_name); request.SetName(table_name); @@ -604,6 +616,11 @@ void GlueCatalog::createNamespaceIfNotExists(const String & namespace_name) cons void GlueCatalog::createTable(const String & namespace_name, const String & table_name, const String & new_metadata_path, Poco::JSON::Object::Ptr /*metadata_content*/) const { + if (!isNamespaceAllowed(namespace_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Failed to create table {}, namespace {} is filtered by `namespaces` database parameter", + table_name, namespace_name); + createNamespaceIfNotExists(namespace_name); Aws::Glue::Model::CreateTableRequest request; @@ -686,6 +703,11 @@ bool GlueCatalog::updateSchema( void GlueCatalog::dropTable(const String & namespace_name, const String & table_name) const { + if (!isNamespaceAllowed(namespace_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Failed to drop table {}, namespace {} is filtered by `namespaces` database parameter", + table_name, namespace_name); + Aws::Glue::Model::DeleteTableRequest request; request.SetDatabaseName(namespace_name); request.SetName(table_name); @@ -699,6 +721,11 @@ void GlueCatalog::dropTable(const String & namespace_name, const String & table_ response.GetError().GetMessage()); } +bool GlueCatalog::isNamespaceAllowed(const std::string & namespace_) const +{ + return allowed_namespaces.contains("*") || allowed_namespaces.contains(namespace_); +} + } #endif diff --git a/src/Databases/DataLake/GlueCatalog.h b/src/Databases/DataLake/GlueCatalog.h index 919b13a5669f..396b7be488bb 100644 --- a/src/Databases/DataLake/GlueCatalog.h +++ b/src/Databases/DataLake/GlueCatalog.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,9 @@ class GlueCatalog final : public ICatalog, private DB::WithContext std::string region; CatalogSettings settings; DB::ASTPtr table_engine_definition; + std::unordered_set allowed_namespaces; + + bool isNamespaceAllowed(const std::string & namespace_) const; DataLake::ICatalog::Namespaces getDatabases(const std::string & prefix, size_t limit = 0) const; DB::Names getTablesForDatabase(const std::string & db_name, size_t limit = 0) const; diff --git a/src/Databases/DataLake/ICatalog.h b/src/Databases/DataLake/ICatalog.h index e14b00ac3732..357f6cd14daf 100644 --- a/src/Databases/DataLake/ICatalog.h +++ b/src/Databases/DataLake/ICatalog.h @@ -128,6 +128,7 @@ struct CatalogSettings String aws_access_key_id; String aws_secret_access_key; String region; + String namespaces; String aws_role_arn; String aws_role_session_name; String aws_external_id; diff --git a/src/Databases/DataLake/RestCatalog.cpp b/src/Databases/DataLake/RestCatalog.cpp index 9f85e9d80d6d..72f44ad54f96 100644 --- a/src/Databases/DataLake/RestCatalog.cpp +++ b/src/Databases/DataLake/RestCatalog.cpp @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -56,6 +57,7 @@ namespace DB::ErrorCodes extern const int LOGICAL_ERROR; extern const int BAD_ARGUMENTS; extern const int FAULT_INJECTED; + extern const int CATALOG_NAMESPACE_DISABLED; } namespace DB::Setting @@ -170,6 +172,7 @@ RestCatalog::RestCatalog( const std::string & auth_header_, const std::string & oauth_server_uri_, bool oauth_server_use_request_body_, + const std::string & namespaces_, DB::ContextPtr context_) : ICatalog(warehouse_) , DB::WithContext(context_) @@ -178,6 +181,7 @@ RestCatalog::RestCatalog( , auth_scope(auth_scope_) , oauth_server_uri(oauth_server_uri_) , oauth_server_use_request_body(oauth_server_use_request_body_) + , allowed_namespaces(namespaces_) { if (!catalog_credential_.empty()) { @@ -213,6 +217,7 @@ RestCatalog::RestCatalog( , auth_scope(auth_scope_) , oauth_server_uri(oauth_server_uri_) , oauth_server_use_request_body(oauth_server_use_request_body_) + , allowed_namespaces("*") { } @@ -643,6 +648,9 @@ bool RestCatalog::empty() const if (found_table) return true; + if (!allowed_namespaces.isNamespaceAllowed(namespace_name, /*nested*/ false)) + return false; + const auto tables = getTables(namespace_name, /* limit */1); if (!tables.empty()) found_table = true; @@ -668,6 +676,8 @@ DB::Names RestCatalog::getTables() const auto execute_for_each_namespace = [&](const std::string & current_namespace) { + if (!allowed_namespaces.isNamespaceAllowed(current_namespace, /*nested*/ false)) + return; runner.enqueueAndKeepTrack( [=, &tables, &mutex, this] { @@ -717,9 +727,21 @@ void RestCatalog::getNamespacesRecursive( break; if (func) - func(current_namespace); + { + if (allowed_namespaces.isNamespaceAllowed(current_namespace, /*nested*/ false)) + func(current_namespace); + else + { + LOG_DEBUG(log, "Tables in namespace {} are filtered", current_namespace); + } + } - getNamespacesRecursive(current_namespace, result, stop_condition, func); + if (allowed_namespaces.isNamespaceAllowed(current_namespace, /*nested*/ true)) + getNamespacesRecursive(current_namespace, result, stop_condition, func); + else + { + LOG_DEBUG(log, "Nested namespaces in namespace {} are filtered", current_namespace); + } } } @@ -894,6 +916,10 @@ RestCatalog::Namespaces RestCatalog::parseNamespaces(DB::ReadBuffer & buf, const DB::Names RestCatalog::getTables(const std::string & base_namespace, size_t limit) const { + if (!allowed_namespaces.isNamespaceAllowed(base_namespace, /*nested*/ false)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Namespace {} is filtered by `namespaces` database parameter", base_namespace); + auto encoded_namespace = encodeNamespaceForURI(base_namespace); const std::string endpoint = std::filesystem::path(NAMESPACES_ENDPOINT) / encoded_namespace / "tables"; @@ -1017,6 +1043,8 @@ bool RestCatalog::tryGetTableMetadata( } catch (const DB::Exception & ex) { + if (ex.code() == DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED) + throw; LOG_DEBUG(log, "tryGetTableMetadata response: {}", ex.what()); return false; } @@ -1038,6 +1066,10 @@ bool RestCatalog::getTableMetadataImpl( { LOG_DEBUG(log, "Checking table {} in namespace {}", table_name, namespace_name); + if (!allowed_namespaces.isNamespaceAllowed(namespace_name, /*nested*/ false)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Namespace {} is filtered by `namespaces` database parameter", namespace_name); + DB::HTTPHeaderEntries headers; if (result.requiresCredentials()) { @@ -1197,6 +1229,10 @@ void RestCatalog::createNamespaceIfNotExists(const String & namespace_name, cons void RestCatalog::createTable(const String & namespace_name, const String & table_name, const String & /*new_metadata_path*/, Poco::JSON::Object::Ptr metadata_content) const { + if (!allowed_namespaces.isNamespaceAllowed(namespace_name, /*nested*/ false)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Failed to create table {}, namespace {} is filtered by `namespaces` database parameter", table_name, namespace_name); + createNamespaceIfNotExists(namespace_name, metadata_content->getValue("location")); const std::string endpoint = (base_url / config.prefix / NAMESPACES_ENDPOINT / encodeNamespaceForURI(namespace_name) / "tables").generic_string(); @@ -1368,6 +1404,11 @@ bool RestCatalog::updateSchema( void RestCatalog::dropTable(const String & namespace_name, const String & table_name) const { + if (!allowed_namespaces.isNamespaceAllowed(namespace_name, /*nested*/ false)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, + "Failed to drop table {}, namespace {} is filtered by `namespaces` database parameter", + table_name, namespace_name); + const std::string endpoint = fmt::format("{}/namespaces/{}/tables/{}?purgeRequested=False", base_url, namespace_name, table_name); Poco::JSON::Object::Ptr request_body = nullptr; @@ -1504,6 +1545,70 @@ ICatalog::CredentialsRefreshCallback RestCatalog::getCredentialsConfigurationCal }; } +/// "alpha,alpha.a1,bravo,bravo.*,charlie,delta.d1,echo.*" +/// allows tables from +/// - "alpha" namespace +/// - "alpha.a1" namespace +/// - "bravo" namespace +/// - any nested namespaces of "bravo" +/// - "charlie" namespace, but not from nested of "charlie" +/// - "delta.d1" namespace, but not from "delta" +/// - any nested namespaces of "echo", but not "echo" itself +/// "bravo.*.b2" makes no sense for now, asterisk allows all nested +RestCatalog::AllowedNamespaces::AllowedNamespaces(const std::string & namespaces_) +{ + std::vector list_of_namespaces; + boost::split(list_of_namespaces, namespaces_, boost::is_any_of(", "), boost::token_compress_on); + for (const auto & ns : list_of_namespaces) + { + std::vector list_of_nested_namespaces; + boost::split(list_of_nested_namespaces, ns, boost::is_any_of(".")); + + size_t len = list_of_nested_namespaces.size(); + if (!len) + continue; + + AllowedNamespaces * current = &(nested_namespaces[list_of_nested_namespaces[0]]); + for (size_t i = 1; i <= len; ++i) + { + if (i == len) + current->allow_tables = true; + else + { + current = &(current->nested_namespaces[list_of_nested_namespaces[i]]); + if (list_of_nested_namespaces[i] == "*") + { + current->allow_tables = true; + break; + } + } + } + } +} + +bool RestCatalog::AllowedNamespaces::isNamespaceAllowed(const std::string & namespace_, bool nested) const +{ + // Trivial case, check here to avoid split namespace on nested + if (nested_namespaces.contains("*")) + return true; + + std::vector list_of_nested_namespaces; + boost::split(list_of_nested_namespaces, namespace_, boost::is_any_of(".")); + + const AllowedNamespaces * current = this; + for (const auto & nns : list_of_nested_namespaces) + { + if (current->nested_namespaces.contains("*")) + return true; + auto it = current->nested_namespaces.find(nns); + if (it == current->nested_namespaces.end()) + return false; + current = &(it->second); + } + + return nested ? !current->nested_namespaces.empty() : current->allow_tables; +} + } #endif diff --git a/src/Databases/DataLake/RestCatalog.h b/src/Databases/DataLake/RestCatalog.h index 982475ee2c96..3a6739026636 100644 --- a/src/Databases/DataLake/RestCatalog.h +++ b/src/Databases/DataLake/RestCatalog.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -43,6 +44,7 @@ class RestCatalog : public ICatalog, public DB::WithContext const std::string & auth_header_, const std::string & oauth_server_uri_, bool oauth_server_use_request_body_, + const std::string & namespaces_, DB::ContextPtr context_); ~RestCatalog() override = default; @@ -131,6 +133,26 @@ class RestCatalog : public ICatalog, public DB::WithContext bool oauth_server_use_request_body; mutable MultiVersion access_token; +public: + class AllowedNamespaces + { + public: + AllowedNamespaces() {} + explicit AllowedNamespaces(const std::string & namespaces_); + + /// Check if nested namespaces (nesetd=true) or tables (nested=false) are allowed in namespace + bool isNamespaceAllowed(const std::string & namespace_, bool nested) const; + + private: + /// List of allowed nested namespaces + std::unordered_map nested_namespaces; + /// Tables from current level are allowed + bool allow_tables = false; + }; + +protected: + AllowedNamespaces allowed_namespaces; + Poco::Net::HTTPBasicCredentials credentials{}; DB::ReadWriteBufferFromHTTPPtr createReadBuffer( diff --git a/src/Databases/DataLake/UnityCatalog.cpp b/src/Databases/DataLake/UnityCatalog.cpp index 414b7e439ecf..c81f03d78d6f 100644 --- a/src/Databases/DataLake/UnityCatalog.cpp +++ b/src/Databases/DataLake/UnityCatalog.cpp @@ -14,12 +14,14 @@ #include #include #include +#include namespace DB::ErrorCodes { extern const int DATALAKE_DATABASE_ERROR; extern const int LOGICAL_ERROR; extern const int BAD_ARGUMENTS; + extern const int CATALOG_NAMESPACE_DISABLED; } namespace @@ -162,6 +164,9 @@ bool UnityCatalog::tryGetTableMetadata( const std::string & table_name, TableMetadata & result) const { + if (!isNamespaceAllowed(schema_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, "Namespace {} is filtered by `namespaces` database parameter", schema_name); + auto full_table_name = warehouse + "." + schema_name + "." + table_name; Poco::Dynamic::Var json; std::string json_str; @@ -284,6 +289,9 @@ bool UnityCatalog::tryGetTableMetadata( bool UnityCatalog::existsTable(const std::string & schema_name, const std::string & table_name) const { + if (!isNamespaceAllowed(schema_name)) + throw DB::Exception(DB::ErrorCodes::CATALOG_NAMESPACE_DISABLED, "Namespace {} is filtered by `namespaces` database parameter", schema_name); + String json_str; Poco::Dynamic::Var json; try @@ -393,7 +401,7 @@ DataLake::ICatalog::Namespaces UnityCatalog::getSchemas(const std::string & base chassert(schema_info->get("catalog_name").extract() == warehouse); UnityCatalogFullSchemaName schema_name = parseFullSchemaName(schema_info->get("full_name").extract()); - if (schema_name.schema_name.starts_with(base_prefix)) + if (isNamespaceAllowed(schema_name.schema_name) && schema_name.schema_name.starts_with(base_prefix)) schemas.push_back(schema_name.schema_name); if (limit && schemas.size() > limit) @@ -435,6 +443,7 @@ UnityCatalog::UnityCatalog( const std::string & catalog_, const std::string & base_url_, const std::string & catalog_credential_, + const std::string & namespaces_, DB::ContextPtr context_) : ICatalog(catalog_) , DB::WithContext(context_) @@ -442,6 +451,12 @@ UnityCatalog::UnityCatalog( , log(getLogger("UnityCatalog(" + catalog_ + ")")) , auth_header("Authorization", "Bearer " + catalog_credential_) { + boost::split(allowed_namespaces, namespaces_, boost::is_any_of(", "), boost::token_compress_on); +} + +bool UnityCatalog::isNamespaceAllowed(const std::string & namespace_) const +{ + return allowed_namespaces.contains("*") || allowed_namespaces.contains(namespace_); } /// getCredentialsConfigurationCallback method is supported only for S3 storage diff --git a/src/Databases/DataLake/UnityCatalog.h b/src/Databases/DataLake/UnityCatalog.h index d835ee221f78..2979e94e124e 100644 --- a/src/Databases/DataLake/UnityCatalog.h +++ b/src/Databases/DataLake/UnityCatalog.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -22,6 +23,7 @@ class UnityCatalog final : public ICatalog, private DB::WithContext const std::string & catalog_, const std::string & base_url_, const std::string & catalog_credential_, + const std::string & namespaces_, DB::ContextPtr context_); ~UnityCatalog() override = default; @@ -60,6 +62,10 @@ class UnityCatalog final : public ICatalog, private DB::WithContext Poco::Net::HTTPBasicCredentials credentials{}; + std::unordered_set allowed_namespaces; + + bool isNamespaceAllowed(const std::string & namespace_) const; + DataLake::ICatalog::Namespaces getSchemas(const std::string & base_prefix, size_t limit = 0) const; DB::Names getTablesForSchema(const std::string & schema, size_t limit = 0) const; diff --git a/src/Databases/DataLake/tests/gtest_rest_catalog.cpp b/src/Databases/DataLake/tests/gtest_rest_catalog.cpp index eb1fd0db1a5c..e385d8b0629b 100644 --- a/src/Databases/DataLake/tests/gtest_rest_catalog.cpp +++ b/src/Databases/DataLake/tests/gtest_rest_catalog.cpp @@ -198,6 +198,7 @@ bool restCatalogEmpty(CatalogShape shape) /* auth_header */"", /* oauth_server_uri */"", /* oauth_server_use_request_body */false, + /* namespaces */"*", context); return catalog.empty(); diff --git a/src/Databases/DataLake/tests/gtest_rest_catalog_allowed_namespaces.cpp b/src/Databases/DataLake/tests/gtest_rest_catalog_allowed_namespaces.cpp new file mode 100644 index 000000000000..7a1981511c3f --- /dev/null +++ b/src/Databases/DataLake/tests/gtest_rest_catalog_allowed_namespaces.cpp @@ -0,0 +1,69 @@ +#include +#include + + +TEST(TestRestCatalogAllowedNamespaces, TestAllAllowed) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces("*"); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); +} + +TEST(TestRestCatalogAllowedNamespaces, TestAllBlocked) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces(""); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); +} + +TEST(TestRestCatalogAllowedNamespaces, TestTableInNamespaceAllowed) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces("foo"); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ false)); +} + +TEST(TestRestCatalogAllowedNamespaces, TestSpecificNestedNamespaceAllowed) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces("foo.bar"); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("bar", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("bar", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.biz", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo.biz", /*nested*/ false)); +} + +TEST(TestRestCatalogAllowedNamespaces, TestNestedNamespacesAllowed) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces("foo.*"); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ false)); +} + +TEST(TestRestCatalogAllowedNamespaces, TestTablesAndNestedNamespacesAllowed) +{ + DataLake::RestCatalog::AllowedNamespaces namespaces("foo,foo.*"); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo", /*nested*/ false)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ true)); + EXPECT_TRUE(namespaces.isNamespaceAllowed("foo.bar", /*nested*/ false)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ true)); + EXPECT_FALSE(namespaces.isNamespaceAllowed("biz", /*nested*/ false)); +} diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/AzureObjectStorage.cpp b/src/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/AzureObjectStorage.cpp index aac0ef6a094a..5c1aa825040a 100644 --- a/src/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/AzureObjectStorage.cpp +++ b/src/Disks/DiskObjectStorage/ObjectStorages/AzureBlobStorage/AzureObjectStorage.cpp @@ -107,6 +107,7 @@ class AzureIteratorAsync final : public IObjectStorageIteratorAsync .etag = blob.Details.ETag.ToString(), .tags = {}, .attributes = {}, + .part_offsets = {}, })); } @@ -216,6 +217,7 @@ void AzureObjectStorage::listObjects(const std::string & path, RelativePathsWith .etag = blob.Details.ETag.ToString(), .tags = {}, .attributes = {}, + .part_offsets = {}, })); } diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/HDFS/HDFSObjectStorage.cpp b/src/Disks/DiskObjectStorage/ObjectStorages/HDFS/HDFSObjectStorage.cpp index e071b25f6b0f..82c8b2779376 100644 --- a/src/Disks/DiskObjectStorage/ObjectStorages/HDFS/HDFSObjectStorage.cpp +++ b/src/Disks/DiskObjectStorage/ObjectStorages/HDFS/HDFSObjectStorage.cpp @@ -269,6 +269,7 @@ void HDFSObjectStorage::listObjects(const std::string & path, RelativePathsWithM .etag = {}, .tags = {}, .attributes = {}, + .part_offsets = {}, })); } diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/IObjectStorage.h b/src/Disks/DiskObjectStorage/ObjectStorages/IObjectStorage.h index 41b16b95e50b..8884f4d15b27 100644 --- a/src/Disks/DiskObjectStorage/ObjectStorages/IObjectStorage.h +++ b/src/Disks/DiskObjectStorage/ObjectStorages/IObjectStorage.h @@ -110,6 +110,10 @@ struct ObjectMetadata std::string etag; ObjectAttributes tags; ObjectAttributes attributes; + /// Cumulative start offsets of the object's multipart-upload parts (part i covers + /// [part_offsets[i], part_offsets[i+1])). Populated best-effort via GetObjectAttributes; empty + /// when unknown or single-PUT. Used to align reads to part boundaries. + std::vector part_offsets; }; struct DataLakeObjectMetadata; @@ -120,6 +124,11 @@ struct RelativePathWithMetadata /// Object metadata: size, modification time, etc. std::optional metadata; + /// Optional per-file hint (bytes) for how much of the file tail to read to get the format's + /// footer/metadata. Set by data lakes that already know per-file stats (e.g. Iceberg from the + /// manifest) and consumed by the format reader (see Parquet ReadOptions::footer_metadata_size_hint). + std::optional footer_size_hint; + RelativePathWithMetadata() = default; explicit RelativePathWithMetadata(String relative_path_, std::optional metadata_ = std::nullopt) diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.cpp b/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.cpp new file mode 100644 index 000000000000..7c6fc306ebaf --- /dev/null +++ b/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.cpp @@ -0,0 +1,49 @@ +#include + +#include +#include + +namespace ProfileEvents +{ + extern const Event ObjectStorageIdentityCacheHits; + extern const Event ObjectStorageIdentityCacheMisses; +} + +namespace CurrentMetrics +{ + extern const Metric ObjectStorageIdentityCacheBytes; + extern const Metric ObjectStorageIdentityCacheCells; +} + +namespace DB +{ + +ObjectStorageIdentityCache::ObjectStorageIdentityCache(const String & cache_policy, size_t max_size_in_bytes, double size_ratio) + : Base( + cache_policy, + CurrentMetrics::ObjectStorageIdentityCacheBytes, + CurrentMetrics::ObjectStorageIdentityCacheCells, + max_size_in_bytes, + /*max_count*/ 0, + size_ratio) +{ +} + +std::optional ObjectStorageIdentityCache::tryGet(const String & key) +{ + if (auto mapped = Base::get(key)) + { + ProfileEvents::increment(ProfileEvents::ObjectStorageIdentityCacheHits); + return *mapped; + } + + ProfileEvents::increment(ProfileEvents::ObjectStorageIdentityCacheMisses); + return std::nullopt; +} + +void ObjectStorageIdentityCache::set(const String & key, ObjectStorageIdentity identity) +{ + Base::set(key, std::make_shared(std::move(identity))); +} + +} diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.h b/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.h new file mode 100644 index 000000000000..152a05c8edeb --- /dev/null +++ b/src/Disks/DiskObjectStorage/ObjectStorages/ObjectStorageIdentityCache.h @@ -0,0 +1,71 @@ +#pragma once + +#include +#include + +#include +#include + +namespace DB +{ + +/// Object-storage "identity": the bits needed to open an object without a fresh HEAD - its size, +/// ETag, and (if multipart-uploaded) the byte offsets of its parts. +/// +/// It exists to break the ordering that forces a HEAD before every open: object caches key on +/// (path, etag), so the etag must be known up front. Here we key by path and store the etag in the +/// value, learned once (via GetObjectAttributes / a GET response) and reused. Multipart part offsets +/// ride along so reads can be aligned to part boundaries without re-probing. +struct ObjectStorageIdentity +{ + String etag; + UInt64 size = 0; + bool is_size_known = true; + /// Cumulative start offset of each multipart part (part i covers [part_offsets[i], part_offsets[i+1])). + /// Empty when unknown or the object is a single PUT. + std::vector part_offsets; + + size_t memoryUsage() const + { + return sizeof(ObjectStorageIdentity) + etag.capacity() + part_offsets.capacity() * sizeof(UInt64); + } +}; + +/// Approximate per-entry weight (bytes) for size-bounded eviction. +struct ObjectStorageIdentityWeight +{ + /// Extra bytes spent on the key string, hashmap node, list links, shared_ptr, etc. + static constexpr size_t OVERHEAD = 128; + + size_t operator()(const ObjectStorageIdentity & identity) const + { + return identity.memoryUsage() + OVERHEAD; + } +}; + +/// Cache of object-storage identity, keyed by object path, so opening an object does not require a +/// fresh HEAD / GetObjectAttributes on every read (a lake scan otherwise re-HEADs every candidate +/// file on every query). Backed by CacheBase (LRU/SLRU) for size-bounded eviction and thread-safety. +/// +/// It is owned by the global Context (see Context::getObjectStorageIdentityCache), sized from server +/// settings, and cleared by `SYSTEM DROP OBJECT STORAGE IDENTITY CACHE`. Low-level object-storage +/// code reaches it via Context::getGlobalContextInstance() and must tolerate a null pointer (no +/// global context yet) by simply issuing the metadata request uncached. +class ObjectStorageIdentityCache + : public CacheBase, ObjectStorageIdentityWeight> +{ + using Base = CacheBase, ObjectStorageIdentityWeight>; + +public: + ObjectStorageIdentityCache(const String & cache_policy, size_t max_size_in_bytes, double size_ratio); + + /// Look up an identity by path. Increments the hit/miss ProfileEvents. + std::optional tryGet(const String & key); + + /// Insert (or overwrite) the identity for a path. + void set(const String & key, ObjectStorageIdentity identity); +}; + +using ObjectStorageIdentityCachePtr = std::shared_ptr; + +} diff --git a/src/Disks/DiskObjectStorage/ObjectStorages/S3/S3ObjectStorage.cpp b/src/Disks/DiskObjectStorage/ObjectStorages/S3/S3ObjectStorage.cpp index 34eb1eaebb9d..95af100aaac9 100644 --- a/src/Disks/DiskObjectStorage/ObjectStorages/S3/S3ObjectStorage.cpp +++ b/src/Disks/DiskObjectStorage/ObjectStorages/S3/S3ObjectStorage.cpp @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -200,6 +201,7 @@ class S3IteratorAsync final : public IObjectStorageIteratorAsync .etag = object.GetETag(), .tags = {}, .attributes = {}, + .part_offsets = {}, }; if (with_tags) metadata.tags = S3::getObjectTags(*client, request->GetBucket(), object.GetKey()); @@ -381,6 +383,7 @@ void S3ObjectStorage::listObjects(const std::string & path, RelativePathsWithMet .etag = object.GetETag(), .tags = {}, .attributes = {}, + .part_offsets = {}, })); if (max_keys) @@ -535,11 +538,50 @@ std::optional S3ObjectStorage::tryGetObjectMetadata(const std::s ObjectMetadata S3ObjectStorage::getObjectMetadata(const std::string & path, bool with_tags) const { + /// When the caller only needs identity (size / etag / part offsets, not tags or user metadata) we + /// take a fast path: consult the process-wide identity cache to skip the metadata request entirely + /// on repeat opens, and on a miss fetch via GetObjectAttributes (one request for size + etag + + /// multipart part sizes, with a HEAD fallback) instead of a plain HEAD. This is what removes the + /// per-file HEAD on lake scans. The tags path is unchanged. See ObjectStorageIdentityCache. + const bool identity_only = !with_tags; + const String identity_key = uri.bucket + "/" + path; + + /// The identity cache is owned by the global Context and sized from server settings. Low-level + /// object-storage code can run before/without a global context (e.g. early startup, standalone + /// disk use); a null pointer simply means "no cache" and we fall back to an uncached request. + ObjectStorageIdentityCachePtr identity_cache; + if (identity_only) + { + if (auto global_context = Context::getGlobalContextInstance()) + identity_cache = global_context->getObjectStorageIdentityCache(); + + if (identity_cache) + { + if (auto cached = identity_cache->tryGet(identity_key)) + { + ObjectMetadata result; + result.size_bytes = cached->size; + result.is_size_known = cached->is_size_known; + result.etag = cached->etag; + result.part_offsets = cached->part_offsets; + return result; + } + } + } + auto settings_ptr = s3_settings.get(); + + auto fetch = [&](const S3::Client & c) -> S3::ObjectInfo + { + if (identity_only) + return S3::getObjectIdentity(c, uri.bucket, path, /*version_id=*/ {}); + return S3::getObjectInfo(c, uri.bucket, path, /*version_id=*/ {}, /*with_metadata=*/ true, /*with_tags=*/ with_tags); + }; + S3::ObjectInfo object_info; try { - object_info = S3::getObjectInfo(*client.get(), uri.bucket, path, /*version_id=*/ {}, /*with_metadata=*/ true, /*with_tags=*/ with_tags); + object_info = fetch(*client.get()); } catch (DB::Exception & e) { @@ -550,7 +592,7 @@ ObjectMetadata S3ObjectStorage::getObjectMetadata(const std::string & path, bool if (new_client) { client.set(std::move(new_client)); - object_info = S3::getObjectInfo(*client.get(), uri.bucket, path, /*version_id=*/ {}, /*with_metadata=*/ true, /*with_tags=*/ with_tags); + object_info = fetch(*client.get()); updated = true; } } @@ -569,6 +611,23 @@ ObjectMetadata S3ObjectStorage::getObjectMetadata(const std::string & path, bool result.tags = std::move(object_info.tags); result.attributes = object_info.metadata; + /// GetObjectAttributes returns per-part sizes; convert to cumulative start offsets. + if (!object_info.part_sizes.empty()) + { + result.part_offsets.reserve(object_info.part_sizes.size()); + uint64_t offset = 0; + for (size_t part_size : object_info.part_sizes) + { + result.part_offsets.push_back(offset); + offset += part_size; + } + } + + if (identity_cache) + identity_cache->set( + identity_key, + ObjectStorageIdentity{result.etag, result.size_bytes, result.is_size_known, result.part_offsets}); + return result; } diff --git a/src/Formats/FormatFactory.cpp b/src/Formats/FormatFactory.cpp index b40fb23ca262..c08bc469cce6 100644 --- a/src/Formats/FormatFactory.cpp +++ b/src/Formats/FormatFactory.cpp @@ -219,6 +219,18 @@ FormatSettings getFormatSettings(const ContextPtr & context, const Settings & se format_settings.parquet.bloom_filter_push_down = settings[Setting::input_format_parquet_bloom_filter_push_down]; format_settings.parquet.page_filter_push_down = settings[Setting::input_format_parquet_page_filter_push_down]; format_settings.parquet.use_offset_index = settings[Setting::input_format_parquet_use_offset_index]; + format_settings.parquet.use_constant_column_optimization = settings[Setting::input_format_parquet_use_constant_column_optimization]; + format_settings.parquet.use_column_index_for_constant_columns = settings[Setting::input_format_parquet_use_column_index_for_constant_columns]; + format_settings.parquet.fill_constant_pages = settings[Setting::input_format_parquet_fill_constant_pages]; + format_settings.parquet.align_reads_to_multipart_boundaries = settings[Setting::input_format_parquet_align_reads_to_multipart_boundaries]; + format_settings.parquet.read_alignment_bytes = settings[Setting::input_format_parquet_read_alignment_bytes]; + format_settings.parquet.read_alignment_min_bytes = settings[Setting::input_format_parquet_read_alignment_min_bytes]; + format_settings.parquet.split_reads_across_part_boundaries = settings[Setting::input_format_parquet_split_reads_across_part_boundaries]; + format_settings.parquet.read_min_fill_ratio = static_cast(settings[Setting::input_format_parquet_read_min_fill_ratio]); + format_settings.parquet.hedged_read_threshold_ms = settings[Setting::input_format_parquet_hedged_read_threshold_ms]; + format_settings.parquet.hedged_read_max_bytes = settings[Setting::input_format_parquet_hedged_read_max_bytes]; + format_settings.parquet.hedged_read_max_inflight = settings[Setting::input_format_parquet_hedged_read_max_inflight]; + format_settings.parquet.prefetch_bandwidth_hide_seconds = settings[Setting::input_format_parquet_prefetch_bandwidth_hide_seconds]; format_settings.parquet.enable_json_parsing = settings[Setting::input_format_parquet_enable_json_parsing]; format_settings.parquet.memory_low_watermark = settings[Setting::input_format_parquet_memory_low_watermark]; @@ -1120,6 +1132,14 @@ bool FormatFactory::checkIfFormatSupportsSubsetOfColumns(const String & name, co return target.subset_of_columns_support_checker && target.subset_of_columns_support_checker(format_settings); } +bool FormatFactory::checkIfFormatIsRandomAccessInput(const String & name) const +{ + if (!exists(name)) + return false; + const auto & target = getCreators(name); + return target.random_access_input_creator || target.random_access_input_creator_with_metadata; +} + void FormatFactory::registerPrewhereSupportChecker(const String & name, PrewhereSupportChecker prewhere_support_checker) { auto & target = getOrCreateCreators(name).prewhere_support_checker; diff --git a/src/Formats/FormatFactory.h b/src/Formats/FormatFactory.h index ce7de7b8bc2b..eb8e763f5688 100644 --- a/src/Formats/FormatFactory.h +++ b/src/Formats/FormatFactory.h @@ -355,6 +355,12 @@ class FormatFactory final : private boost::noncopyable, public IHints<2> bool checkIfOutputFormatPrefersLargeBlocks(const String & name) const; bool checkIfOutputFormatIsTTYFriendly(const String & name) const; + /// True for column-oriented input formats that seek back and forth in the file (Parquet, ORC, + /// Arrow) - i.e. formats registered via registerRandomAccessInputFormat[WithMetadata]. Such + /// formats read the footer at the tail first and drive their own prefetching, so a generic + /// read-ahead from the start of the file is usually counter-productive for them. + bool checkIfFormatIsRandomAccessInput(const String & name) const; + bool checkParallelizeOutputAfterReading(const String & name, const ContextPtr & context) const; void registerAdditionalInfoForSchemaCacheGetter(const String & name, AdditionalInfoForSchemaCacheGetter additional_info_for_schema_cache_getter); diff --git a/src/Formats/FormatSettings.h b/src/Formats/FormatSettings.h index 728a1faba670..1be19293558c 100644 --- a/src/Formats/FormatSettings.h +++ b/src/Formats/FormatSettings.h @@ -349,6 +349,18 @@ struct FormatSettings bool bloom_filter_push_down = true; bool page_filter_push_down = true; bool use_offset_index = true; + bool use_constant_column_optimization = true; + bool use_column_index_for_constant_columns = false; + bool fill_constant_pages = false; + bool align_reads_to_multipart_boundaries = false; + UInt64 read_alignment_bytes = 0; + UInt64 read_alignment_min_bytes = 1048576; + bool split_reads_across_part_boundaries = false; + double read_min_fill_ratio = 0; + UInt64 hedged_read_threshold_ms = 0; + UInt64 hedged_read_max_bytes = 4194304; + UInt64 hedged_read_max_inflight = 4; + double prefetch_bandwidth_hide_seconds = 0; bool enable_json_parsing = true; bool preserve_order = false; diff --git a/src/IO/S3/Client.cpp b/src/IO/S3/Client.cpp index af6f0a2e6894..1b4ba03a2945 100644 --- a/src/IO/S3/Client.cpp +++ b/src/IO/S3/Client.cpp @@ -506,6 +506,12 @@ Model::GetObjectTaggingOutcome Client::GetObjectTagging(GetObjectTaggingRequest doRequest(request, [this](const Model::GetObjectTaggingRequest & req) { return GetObjectTagging(req); })); } +Model::GetObjectAttributesOutcome Client::GetObjectAttributes(GetObjectAttributesRequest & request) const +{ + return processRequestResult( + doRequest(request, [this](const Model::GetObjectAttributesRequest & req) { return GetObjectAttributes(req); })); +} + Model::ListObjectsV2Outcome Client::ListObjectsV2(ListObjectsV2Request & request) const { return doRequestWithRetryNetworkErrors( diff --git a/src/IO/S3/Client.h b/src/IO/S3/Client.h index ad4d685d88a2..363c169d81e2 100644 --- a/src/IO/S3/Client.h +++ b/src/IO/S3/Client.h @@ -211,6 +211,7 @@ class Client : private Aws::S3::S3Client Model::ListObjectsV2Outcome ListObjectsV2(ListObjectsV2Request & request) const; Model::ListObjectsOutcome ListObjects(ListObjectsRequest & request) const; Model::GetObjectOutcome GetObject(GetObjectRequest & request) const; + Model::GetObjectAttributesOutcome GetObjectAttributes(GetObjectAttributesRequest & request) const; Model::AbortMultipartUploadOutcome AbortMultipartUpload(AbortMultipartUploadRequest & request) const; Model::CreateMultipartUploadOutcome CreateMultipartUpload(CreateMultipartUploadRequest & request) const; @@ -276,6 +277,7 @@ class Client : private Aws::S3::S3Client using Aws::S3::S3Client::ListObjectsV2; using Aws::S3::S3Client::ListObjects; using Aws::S3::S3Client::GetObject; + using Aws::S3::S3Client::GetObjectAttributes; using Aws::S3::S3Client::AbortMultipartUpload; using Aws::S3::S3Client::CreateMultipartUpload; diff --git a/src/IO/S3/PocoHTTPClient.cpp b/src/IO/S3/PocoHTTPClient.cpp index 2b76300bbbc9..90f59b4b469e 100644 --- a/src/IO/S3/PocoHTTPClient.cpp +++ b/src/IO/S3/PocoHTTPClient.cpp @@ -14,6 +14,7 @@ #include #include +#include #include #include #include @@ -645,7 +646,21 @@ void PocoHTTPClient::makeRequestInternalImpl( if (enable_s3_requests_logging) LOG_TEST(log, "Receiving response..."); + /// Measure time-to-first-byte: the wait for and read of the response status line and + /// headers. This isolates request latency (network RTT + server/backend first-byte + /// time) from the connect and body-transfer phases, and is fed back per resolved + /// address to drive latency-aware host selection when it is enabled. + Stopwatch first_byte_watch; auto & response_body_stream = session->receiveResponse(poco_response); + const UInt64 first_byte_us = first_byte_watch.elapsedMicroseconds(); + + if (HostResolver::latencyAwareSelectionEnabled() && proxy_configuration.isEmpty()) + { + Poco::Net::IPAddress resolved_ip; + const std::string resolved_host = session->getResolvedHost(); + if (!resolved_host.empty() && Poco::Net::IPAddress::tryParse(resolved_host, resolved_ip)) + HostResolversPool::instance().getResolver(target_uri.getHost())->reportLatency(resolved_ip, first_byte_us); + } watch.stop(); addMetric(request, S3MetricType::Microseconds, watch.elapsedMicroseconds()); diff --git a/src/IO/S3/Requests.h b/src/IO/S3/Requests.h index aa21602674f0..1db171fa7a3e 100644 --- a/src/IO/S3/Requests.h +++ b/src/IO/S3/Requests.h @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -157,6 +158,7 @@ class HeadObjectRequest: public ExtendedRequest using ListObjectsV2Request = ExtendedRequest; using ListObjectsRequest = ExtendedRequest; using GetObjectRequest = ExtendedRequest; +using GetObjectAttributesRequest = ExtendedRequest; using GetObjectTaggingRequest = ExtendedRequest; class UploadPartRequest : public ExtendedRequest diff --git a/src/IO/S3/getObjectInfo.cpp b/src/IO/S3/getObjectInfo.cpp index 1cdcd9f94e37..74f1517cc440 100644 --- a/src/IO/S3/getObjectInfo.cpp +++ b/src/IO/S3/getObjectInfo.cpp @@ -4,12 +4,16 @@ #if USE_AWS_S3 +#include + namespace ProfileEvents { extern const Event S3GetObjectTagging; extern const Event S3HeadObject; + extern const Event S3GetObjectAttributes; extern const Event DiskS3GetObjectTagging; extern const Event DiskS3HeadObject; + extern const Event DiskS3GetObjectAttributes; } @@ -38,6 +42,39 @@ namespace return client.HeadObject(req); } + Aws::S3::Model::GetObjectAttributesOutcome getObjectAttributes( + const S3::Client & client, + const String & bucket, + const String & key, + const String & version_id) + { + ProfileEvents::increment(ProfileEvents::S3GetObjectAttributes); + if (client.isClientForDisk()) + ProfileEvents::increment(ProfileEvents::DiskS3GetObjectAttributes); + + S3::GetObjectAttributesRequest req; + req.SetBucket(bucket); + req.SetKey(key); + if (!version_id.empty()) + req.SetVersionId(version_id); + req.SetObjectAttributes({ + Aws::S3::Model::ObjectAttributes::ETag, + Aws::S3::Model::ObjectAttributes::ObjectSize, + Aws::S3::Model::ObjectAttributes::ObjectParts}); + + return client.GetObjectAttributes(req); + } + + /// GetObjectAttributes returns the ETag without surrounding quotes, whereas HeadObject returns it + /// quoted. Normalize (strip quotes) so identities are consistent regardless of which path produced + /// them - important because they are used as cache keys. + String stripQuotes(String s) + { + if (s.size() >= 2 && s.front() == '"' && s.back() == '"') + s = s.substr(1, s.size() - 2); + return s; + } + Aws::S3::Model::GetObjectTaggingOutcome getObjectTagging( const S3::Client & client, const String & bucket, @@ -180,6 +217,35 @@ ObjectInfo getObjectInfo( getAuthenticationErrorHint(error.GetErrorType())); } +ObjectInfo getObjectIdentity( + const S3::Client & client, + const String & bucket, + const String & key, + const String & version_id) +{ + { + Expect404ResponseScope scope; // a not-found here just falls through to HEAD below + auto outcome = getObjectAttributes(client, bucket, key, version_id); + if (outcome.IsSuccess()) + { + const auto & result = outcome.GetResult(); + ObjectInfo object_info; + object_info.size = static_cast(result.GetObjectSize()); + object_info.is_size_known = true; + object_info.etag = stripQuotes(result.GetETag()); + for (const auto & part : result.GetObjectParts().GetParts()) + object_info.part_sizes.push_back(static_cast(part.GetSize())); + return object_info; + } + /// GetObjectAttributes unsupported (many S3-compatible stores), denied (distinct IAM action), + /// or otherwise failed - fall back to a plain HEAD. Best-effort: we lose part_sizes only. + } + + ObjectInfo object_info = getObjectInfo(client, bucket, key, version_id, /*with_metadata=*/ false, /*with_tags=*/ false); + object_info.etag = stripQuotes(object_info.etag); + return object_info; +} + size_t getObjectSize( const S3::Client & client, const String & bucket, diff --git a/src/IO/S3/getObjectInfo.h b/src/IO/S3/getObjectInfo.h index 314d81c6daa5..0c5455910d76 100644 --- a/src/IO/S3/getObjectInfo.h +++ b/src/IO/S3/getObjectInfo.h @@ -3,6 +3,7 @@ #include "config.h" #if USE_AWS_S3 +#include #include #include #include @@ -19,6 +20,10 @@ struct ObjectInfo String etag; ObjectAttributes tags; // Set only if getObjectInfo() is called with `with_tags = true` ObjectAttributes metadata = {}; /// Set only if getObjectInfo() is called with `with_metadata = true`. + /// Sizes of the object's multipart-upload parts (offsets = prefix sums). Set only by + /// getObjectIdentity() when the object was multipart-uploaded and GetObjectAttributes succeeded; + /// empty for single-PUT objects or when the call fell back to HEAD. + std::vector part_sizes; }; /// Ignore if object does not exist @@ -38,6 +43,17 @@ ObjectInfo getObjectInfo( bool with_metadata = false, bool with_tags = false); +/// Fetches size + etag (+ multipart part sizes) in a single GetObjectAttributes request instead of a +/// HEAD, so a caller that also wants the multipart layout gets it without a second round-trip. Falls +/// back to getObjectInfo() (HEAD) if GetObjectAttributes is unsupported, denied, or otherwise errors, +/// so it is safe against S3-compatible stores that lack the API. `part_sizes` is populated only on +/// the GetObjectAttributes path for multipart objects; it is empty on the fallback path. +ObjectInfo getObjectIdentity( + const S3::Client & client, + const String & bucket, + const String & key, + const String & version_id = {}); + ObjectAttributes getObjectTags( const S3::Client & client, const String & bucket, diff --git a/src/Interpreters/Context.cpp b/src/Interpreters/Context.cpp index 1edce5d06b45..9a5b6feab274 100644 --- a/src/Interpreters/Context.cpp +++ b/src/Interpreters/Context.cpp @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -558,6 +559,7 @@ struct ContextSharedPart : boost::noncopyable mutable ResourceManagerPtr resource_manager; mutable UncompressedCachePtr uncompressed_cache TSA_GUARDED_BY(mutex); /// The cache of decompressed blocks. mutable MarkCachePtr mark_cache TSA_GUARDED_BY(mutex); /// Cache of marks in compressed files. + mutable ObjectStorageIdentityCachePtr object_storage_identity_cache TSA_GUARDED_BY(mutex); /// Cache of object-storage identity (size/etag/part-offsets) to avoid per-open HEADs. mutable UniqueKeyIndexCachePtr unique_key_index_cache TSA_GUARDED_BY(mutex); /// RocksDB-compatible block cache over CacheBase for the UNIQUE KEY index (nullptr when RocksDB unavailable or disabled). mutable DeleteBitmapCachePtr delete_bitmap_cache TSA_GUARDED_BY(mutex); /// UNIQUE KEY per-part delete-bitmap cache. mutable PrimaryIndexCachePtr primary_index_cache TSA_GUARDED_BY(mutex); @@ -4101,6 +4103,45 @@ void Context::clearMarkCache() const JemallocCacheArena::purge(); } +void Context::setObjectStorageIdentityCache(const String & cache_policy, size_t max_size_in_bytes, double size_ratio) +{ + std::lock_guard lock(shared->mutex); + + if (shared->object_storage_identity_cache) + throw Exception(ErrorCodes::LOGICAL_ERROR, "Object storage identity cache has been already created."); + + /// A zero size disables the cache: leave the pointer unset so callers fall back to uncached reads. + if (max_size_in_bytes == 0) + return; + + shared->object_storage_identity_cache = std::make_shared(cache_policy, max_size_in_bytes, size_ratio); +} + +void Context::updateObjectStorageIdentityCacheConfiguration(const Poco::Util::AbstractConfiguration & config) +{ + std::lock_guard lock(shared->mutex); + + if (!shared->object_storage_identity_cache) + return; + + size_t size = config.getUInt64("object_storage_identity_cache_size", 67108864); + shared->object_storage_identity_cache->setMaxSizeInBytes(size); +} + +std::shared_ptr Context::getObjectStorageIdentityCache() const +{ + SharedLockGuard lock(shared->mutex); + return shared->object_storage_identity_cache; +} + +void Context::clearObjectStorageIdentityCache() const +{ + ObjectStorageIdentityCachePtr cache = getObjectStorageIdentityCache(); + + if (cache) + cache->clear(); +} + void Context::setUniqueKeyIndexCache( [[maybe_unused]] const String & cache_policy, [[maybe_unused]] size_t max_cache_size_in_bytes, diff --git a/src/Interpreters/Context.h b/src/Interpreters/Context.h index e939d24e0e86..b29a0887ed23 100644 --- a/src/Interpreters/Context.h +++ b/src/Interpreters/Context.h @@ -101,6 +101,7 @@ class RefreshSet; class Cluster; class Compiler; class MarkCache; +class ObjectStorageIdentityCache; class UniqueKeyIndexCache; class DeleteBitmapCache; class PrimaryIndexCache; @@ -1420,6 +1421,13 @@ class Context: public ContextData, public std::enable_shared_from_this void clearMarkCache() const; ThreadPool & getLoadMarksThreadpool() const; + /// Object-storage identity cache (per-object size/ETag/multipart-part-offsets) to avoid a HEAD + /// on every object open. A max_size_in_bytes of 0 leaves the cache unset (disabled). + void setObjectStorageIdentityCache(const String & cache_policy, size_t max_size_in_bytes, double size_ratio); + void updateObjectStorageIdentityCacheConfiguration(const Poco::Util::AbstractConfiguration & config); + std::shared_ptr getObjectStorageIdentityCache() const; + void clearObjectStorageIdentityCache() const; + /// UNIQUE KEY index cache: ClickHouse-side `CacheBase` adapter /// over the RocksDB block cache used by SST-backed UNIQUE KEY indexes. void setUniqueKeyIndexCache(const String & cache_policy, size_t max_cache_size_in_bytes, double size_ratio); diff --git a/src/Interpreters/InterpreterSystemQuery.cpp b/src/Interpreters/InterpreterSystemQuery.cpp index 8220a1a82db6..1d533206c877 100644 --- a/src/Interpreters/InterpreterSystemQuery.cpp +++ b/src/Interpreters/InterpreterSystemQuery.cpp @@ -481,6 +481,10 @@ BlockIO InterpreterSystemQuery::execute() #else throw Exception(ErrorCodes::SUPPORT_IS_DISABLED, "The server was compiled without the support for Parquet"); #endif + case Type::CLEAR_OBJECT_STORAGE_IDENTITY_CACHE: + getContext()->checkAccess(AccessType::SYSTEM_DROP_OBJECT_STORAGE_IDENTITY_CACHE); + system_context->clearObjectStorageIdentityCache(); + break; case Type::CLEAR_PRIMARY_INDEX_CACHE: getContext()->checkAccess(AccessType::SYSTEM_DROP_PRIMARY_INDEX_CACHE); system_context->clearPrimaryIndexCache(); @@ -2475,6 +2479,7 @@ AccessRightsElements InterpreterSystemQuery::getRequiredAccessForDDLOnCluster() case Type::CLEAR_ICEBERG_METADATA_CACHE: case Type::CLEAR_AVRO_SCHEMA_CACHE: case Type::CLEAR_PARQUET_METADATA_CACHE: + case Type::CLEAR_OBJECT_STORAGE_IDENTITY_CACHE: case Type::CLEAR_PRIMARY_INDEX_CACHE: case Type::CLEAR_MMAP_CACHE: case Type::CLEAR_QUERY_CONDITION_CACHE: diff --git a/src/Parsers/ASTSystemQuery.cpp b/src/Parsers/ASTSystemQuery.cpp index c4ee97348899..9f38ec1aa592 100644 --- a/src/Parsers/ASTSystemQuery.cpp +++ b/src/Parsers/ASTSystemQuery.cpp @@ -603,6 +603,7 @@ void ASTSystemQuery::formatImpl(WriteBuffer & ostr, const FormatSettings & setti case Type::CLEAR_S3_CLIENT_CACHE: case Type::CLEAR_ICEBERG_METADATA_CACHE: case Type::CLEAR_PARQUET_METADATA_CACHE: + case Type::CLEAR_OBJECT_STORAGE_IDENTITY_CACHE: case Type::CLEAR_AVRO_SCHEMA_CACHE: case Type::RESET_COVERAGE: case Type::RESTART_REPLICAS: diff --git a/src/Parsers/ASTSystemQuery.h b/src/Parsers/ASTSystemQuery.h index adf76d898d42..333562e08595 100644 --- a/src/Parsers/ASTSystemQuery.h +++ b/src/Parsers/ASTSystemQuery.h @@ -44,6 +44,7 @@ class ASTSystemQuery : public IAST, public ASTQueryWithOnCluster CLEAR_COMPILED_EXPRESSION_CACHE, CLEAR_ICEBERG_METADATA_CACHE, CLEAR_PARQUET_METADATA_CACHE, + CLEAR_OBJECT_STORAGE_IDENTITY_CACHE, CLEAR_FILESYSTEM_CACHE, CLEAR_DISTRIBUTED_CACHE, CLEAR_DISK_METADATA_CACHE, diff --git a/src/Parsers/ParserSystemQuery.cpp b/src/Parsers/ParserSystemQuery.cpp index 07b79465ea04..8772885e29d1 100644 --- a/src/Parsers/ParserSystemQuery.cpp +++ b/src/Parsers/ParserSystemQuery.cpp @@ -285,6 +285,7 @@ bool ParserSystemQuery::parseImpl(IParser::Pos & pos, ASTPtr & node, Expected & {"DROP COMPILED EXPRESSION CACHE", Type::CLEAR_COMPILED_EXPRESSION_CACHE}, {"DROP ICEBERG METADATA CACHE", Type::CLEAR_ICEBERG_METADATA_CACHE}, {"DROP PARQUET METADATA CACHE", Type::CLEAR_PARQUET_METADATA_CACHE}, + {"DROP OBJECT STORAGE IDENTITY CACHE", Type::CLEAR_OBJECT_STORAGE_IDENTITY_CACHE}, {"DROP FILESYSTEM CACHE", Type::CLEAR_FILESYSTEM_CACHE}, {"DROP DISTRIBUTED CACHE", Type::CLEAR_DISTRIBUTED_CACHE}, {"DROP DISK METADATA CACHE", Type::CLEAR_DISK_METADATA_CACHE}, diff --git a/src/Processors/Formats/Impl/Parquet/Prefetcher.cpp b/src/Processors/Formats/Impl/Parquet/Prefetcher.cpp index 1141cfe870a2..99e3a37efa17 100644 --- a/src/Processors/Formats/Impl/Parquet/Prefetcher.cpp +++ b/src/Processors/Formats/Impl/Parquet/Prefetcher.cpp @@ -3,12 +3,20 @@ #include #include #include +#include +#include +#include +#include #include #include #include #include #include +#include +#include +#include +#include namespace DB::ErrorCodes { @@ -22,6 +30,15 @@ namespace ProfileEvents extern const Event ParquetPrefetcherReadRandomRead; extern const Event ParquetPrefetcherReadSeekAndRead; extern const Event ParquetPrefetcherReadEntireFile; + extern const Event ParquetPrefetcherServedFromRetainedTail; + extern const Event ParquetPrefetcherPartAlignedTasks; + extern const Event ParquetPrefetcherAlignmentSkippedSmall; + extern const Event ParquetPrefetcherHedgedReads; + extern const Event ParquetPrefetcherHedgedWins; + extern const Event ParquetPrefetcherSplitReadTasks; + extern const Event ParquetPrefetcherSplitReadSegments; + extern const Event ParquetPrefetcherFillRatioLimitedTasks; + extern const Event ParquetPrefetcherFooterSpeculativeParallel; } namespace DB::Parquet @@ -31,9 +48,29 @@ void Prefetcher::init(ReadBuffer * reader_, const ReadOptions & options, FormatP { min_bytes_for_seek = options.min_bytes_for_seek; bytes_per_read_task = options.bytes_per_read_task; + multipart_part_offsets = options.multipart_part_offsets; + read_alignment_stride = options.read_alignment_stride; + read_alignment_min_bytes = options.read_alignment_min_bytes; + split_reads_across_boundaries = options.split_reads_across_boundaries; + read_min_fill_ratio = options.read_min_fill_ratio; + hedged_read_threshold_ms = options.hedged_read_threshold_ms; + hedged_read_max_bytes = options.hedged_read_max_bytes; + hedged_read_max_inflight = options.hedged_read_max_inflight; parser_shared_resources = parser_shared_resources_; determineReadModeAndFileSize(reader_, options); range_sets.resize(1); + read_start_time = std::chrono::steady_clock::now(); +} + +double Prefetcher::averageThroughputBytesPerSec() const +{ + double seconds = std::chrono::duration(std::chrono::steady_clock::now() - read_start_time).count(); + size_t bytes = total_bytes_read.load(std::memory_order_relaxed); + /// Too little read / too little time elapsed to estimate: report "unknown" so back-pressure + /// fails open (does not throttle). + if (seconds < 0.05 || bytes < (1u << 20)) + return 0; + return static_cast(bytes) / seconds; } Prefetcher::~Prefetcher() @@ -124,6 +161,169 @@ void Prefetcher::readSync(char * to, size_t n, size_t offset) throw Exception(ErrorCodes::INCORRECT_DATA, "Unexpected eof: offset {}, length {}, bytes read {}, expected file size {}", offset, n, nread, file_size); } +bool Prefetcher::gapCrossesBoundary(size_t lo, size_t hi) const +{ + /// Does the byte range [lo, hi) contain a part boundary strictly inside (lo, hi)? If so, reading + /// across it (to bridge a coalescing gap) would straddle a part. + if (lo >= hi) + return false; + if (!multipart_part_offsets.empty()) + { + auto it = std::upper_bound(multipart_part_offsets.begin(), multipart_part_offsets.end(), lo); + return it != multipart_part_offsets.end() && *it < hi; + } + if (read_alignment_stride > 0) + return (lo / read_alignment_stride) != ((hi - 1) / read_alignment_stride); + return false; +} + +bool Prefetcher::fillRatioWouldBreak(size_t wanted, size_t start_offset, size_t end_offset, const RangeState & r) const +{ + /// Anti-amplification guard for coalescing. Bridging the gap to range r reads the bytes between it + /// and the current task - cheap for a small gap, but many sub-min_bytes_for_seek gaps accumulate + /// into a task that is almost all filler (e.g. a tiny scattered near-constant column dragging in + /// its neighbours). Refuse to extend if the resulting task's "wanted / span" fraction would drop + /// below read_min_fill_ratio. Tiny gaps are always allowed (below the floor) so dense reads, whose + /// fill stays ~1, are never split. + if (read_min_fill_ratio <= 0) + return false; + + static constexpr size_t small_gap_floor = 64 << 10; + const size_t gap = start_offset > r.end ? start_offset - r.end + : r.start > end_offset ? r.start - end_offset + : 0; + if (gap < small_gap_floor) + return false; + + const size_t new_span = std::max(end_offset, r.end) - std::min(start_offset, r.start); + const size_t new_wanted = wanted + r.length(); + if (new_span == 0) + return false; + + if (static_cast(new_wanted) < read_min_fill_ratio * static_cast(new_span)) + { + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherFillRatioLimitedTasks); + return true; + } + return false; +} + +std::vector Prefetcher::splitPointsForRange(size_t offset, size_t length) const +{ + std::vector pts; + if (length == 0) + return pts; + size_t end = offset + length; + if (!multipart_part_offsets.empty()) + { + /// Real per-file part boundaries (cumulative start offsets). Take those strictly inside. + auto it = std::upper_bound(multipart_part_offsets.begin(), multipart_part_offsets.end(), offset); + for (; it != multipart_part_offsets.end() && *it < end; ++it) + pts.push_back(*it); + } + else if (read_alignment_stride > 0) + { + /// Fixed grid: boundaries at multiples of the stride. + size_t b = (offset / read_alignment_stride + 1) * read_alignment_stride; + for (; b < end; b += read_alignment_stride) + pts.push_back(b); + } + return pts; +} + +void Prefetcher::readSyncParallel(const std::vector> & reads) +{ + if (reads.size() <= 1) + { + for (const auto & [to, n, offset] : reads) + readSync(to, n, offset); + return; + } + + /// Run the segments on a SEPARATE pool (getIOThreadPool), NOT io_runner/parsing_runner (both + /// backed by getFormatParsingThreadPool). This is essential: + /// * it doesn't consume io_runner slots, so splitting never starves the prefetch pipeline; + /// * it's deadlock-safe from any caller - the calling thread (a parsing/io_runner worker, on + /// getFormatParsingThreadPool) blocks on a DIFFERENT pool whose tasks are leaf reads that + /// always drain; + /// * it works even when prefetch (io_runner) is disabled. + ThreadPool * pool = nullptr; + try + { + pool = &getIOThreadPool().get(); + } + catch (...) + { + pool = nullptr; + } + /// Adaptive capacity gate: only parallelize when the separate pool actually has a free thread + /// right now. If it's saturated (by our own or other IO), splitting would just queue the + /// segments behind other work - no parallelism, only extra GETs. In that case do the single + /// sequential read instead. active() reflects all getIOThreadPool users, so this backs off + /// under global IO pressure. (Snapshot/racy, but a good heuristic - being off by one is cheap.) + const size_t max_threads = pool ? pool->getMaxThreads() : 0; + const size_t active = pool ? pool->active() : 0; + const size_t spare = max_threads > active ? max_threads - active : 0; + if (pool == nullptr || max_threads <= 1 || spare == 0) + { + /// No usable separate pool / no free thread: fall back to sequential (correct, not parallel). + for (const auto & [to, n, offset] : reads) + readSync(to, n, offset); + return; + } + + auto runner = threadPoolCallbackRunnerUnsafe(*pool, ThreadName::PARALLEL_READ); + std::vector> futures; + futures.reserve(reads.size() - 1); + for (size_t i = 1; i < reads.size(); ++i) + { + auto [to, n, offset] = reads[i]; + futures.push_back(runner([this, to, n, offset] { readSync(to, n, offset); }, Priority{})); + } + + /// Run the first segment inline on this thread, in parallel with the pool-run ones. + std::exception_ptr first_exception; + try + { + auto [to, n, offset] = reads[0]; + readSync(to, n, offset); + } + catch (...) + { + first_exception = std::current_exception(); + } + + /// Join all segments (must wait for every one before returning, buffers are on our stack/heap). + for (auto & f : futures) + { + try + { + f.get(); + } + catch (...) + { + if (!first_exception) + first_exception = std::current_exception(); + } + } + + if (first_exception) + std::rethrow_exception(first_exception); +} + +void Prefetcher::retainTail(const char * data, size_t length, size_t file_offset) +{ + /// Nothing to save when the whole file is already in memory (getRangeData copies from + /// entire_file, no read is issued). Only the read-issuing modes benefit. + if (read_mode == ReadMode::EntireFileIsInMemory || length == 0) + return; + chassert(!ranges_finalized.load(std::memory_order_relaxed)); + chassert(file_offset + length <= file_size); + retained_tail.assign(data, data + length); + retained_tail_start = file_offset; + retained_tail_end = file_offset + length; +} + PrefetchHandle Prefetcher::registerRange(size_t offset, size_t length, bool likely_to_be_used) { chassert(!ranges_finalized.load(std::memory_order_relaxed)); @@ -300,6 +500,36 @@ void Prefetcher::pickRangesAndCreateTaskIfNotExists(RequestState * initial_req, end_offset = ranges[range_idx].end; } + /// If this range is fully contained in the retained footer tail, serve it directly from that + /// in-memory copy - no read, no coalescing. Build a one-off Task already in the Done state whose + /// cached_region points into `retained_tail`; getRangeData's zero-copy path then returns the + /// span. This eliminates the redundant Column/Offset Index read (their bytes were already + /// fetched for the footer). `retained_tail` outlives all handles (it is a Prefetcher member), so + /// the region needs no keep-alive handle. + if (retained_tail_end > retained_tail_start + && start_offset >= retained_tail_start && end_offset <= retained_tail_end) + { + Task & task = tasks.emplace_back(); + task.offset = start_offset; + task.length = end_offset - start_offset; + task.memory_amplification = 1; + task.refcount.store(1); + task.state.store(Task::State::Done); + task.cached_region = Task::CachedReadRegion{ + .handle = {}, + .data = retained_tail.data() + (start_offset - retained_tail_start), + .size = end_offset - start_offset, + .file_offset = start_offset}; + + initial_req->task = &task; + initial_req->task_offset = 0; + RequestState::State s = RequestState::State::HasRange; + bool ok = initial_req->state.compare_exchange_strong(s, RequestState::State::HasTask); + chassert(ok); // we hold a PrefetchHandle, so it cannot be Cancelled here + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherServedFromRetainedTail); + return; // lock released by unique_lock destructor; task is already Done, nothing to schedule + } + /// Try to extend the task's range in both directions to cover more request ranges, as long /// as gaps between them are shorter than min_bytes_for_seek. @@ -307,6 +537,17 @@ void Prefetcher::pickRangesAndCreateTaskIfNotExists(RequestState * initial_req, size_t end_idx = range_idx + 1; size_t total_length_of_covered_ranges = end_offset - start_offset; + /// EXPERIMENTAL read alignment: never bridge a coalescing gap that crosses a part boundary. + /// Bridging a gap means reading the (unneeded) bytes between two wanted ranges to avoid a seek - + /// cheap when both ends are in the same part, but if the gap spans a boundary the merged read + /// straddles it (pays ~an extra round-trip on S3) AND wastes those bytes. So at a boundary we cut: + /// the gap is never read and no read straddles. Boundaries come from the real multipart layout + /// (multipart_part_offsets) when known, else a fixed grid (read_alignment_stride). Gaps within a + /// single part are still bridged as before. (A single wanted range larger than a part is left as + /// is - splitting that is the job of the read-splitter, not coalescing.) + const bool alignment_active = !multipart_part_offsets.empty() || read_alignment_stride > 0; + bool part_boundary_constrained = false; + /// Go left. size_t initial_offset = start_offset; for (size_t idx = range_idx; idx > 0; --idx) @@ -317,9 +558,20 @@ void Prefetcher::pickRangesAndCreateTaskIfNotExists(RequestState * initial_req, !r.request->allow_incidental_read.load(std::memory_order_relaxed)) // range wants to be coalesced break; + if (alignment_active && gapCrossesBoundary(r.end, start_offset)) // bridging back to r crosses a part boundary + { + part_boundary_constrained = true; + break; + } + const auto s = r.request->state.load(std::memory_order_relaxed); if (s == RequestState::State::HasRange) { + /// Anti-amplification: stop extending if bridging this (non-trivial) gap would make the + /// task mostly filler. Tiny gaps are always bridged so dense reads are unaffected. + if (fillRatioWouldBreak(total_length_of_covered_ranges, start_offset, end_offset, r)) + break; + /// Include this range in the task. start_idx = idx - 1; total_length_of_covered_ranges += r.length(); @@ -352,9 +604,18 @@ void Prefetcher::pickRangesAndCreateTaskIfNotExists(RequestState * initial_req, !r.request->allow_incidental_read.load(std::memory_order_relaxed)) break; + if (alignment_active && gapCrossesBoundary(end_offset, r.start)) // bridging forward to r crosses a part boundary + { + part_boundary_constrained = true; + break; + } + const auto s = r.request->state.load(std::memory_order_relaxed); if (s == RequestState::State::HasRange) { + if (fillRatioWouldBreak(total_length_of_covered_ranges, start_offset, end_offset, r)) + break; + end_idx = idx + 1; total_length_of_covered_ranges += r.length(); end_offset = std::max(end_offset, r.end); @@ -368,6 +629,9 @@ void Prefetcher::pickRangesAndCreateTaskIfNotExists(RequestState * initial_req, } } + if (part_boundary_constrained) + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherPartAlignedTasks); + /// Create task. Task & task = tasks.emplace_back(); task.offset = start_offset; @@ -410,6 +674,7 @@ void Prefetcher::decreaseTaskRefcount(Task * task, size_t amount) { task->buf = {}; task->cached_region.reset(); + task->hedge_buf = {}; // hedge (if any) is synchronous and already finished by now } } @@ -421,10 +686,51 @@ void Prefetcher::scheduleTask(Task * task) std::shared_lock shutdown_lock(*_shutdown, std::try_to_lock); if (!shutdown_lock.owns_lock()) return; - runTask(task); + /// Prefetched reads are NOT split: no consumer is blocked on them (they're ahead of + /// demand), so cutting their latency buys nothing - it would only add GETs. Splitting + /// is reserved for the inline getRangeData path (a consumer is actually waiting). + runTask(task, /*allow_split*/ false); }); } +bool Prefetcher::hedgeReadSync(Task * task) +{ + /// Wait up to the threshold for the primary; if it finishes in time, no hedge is needed. + if (task->completion.wait_for(hedged_read_threshold_ms)) + return false; + + /// Primary is slow. Only one consumer hedges a given task. + bool expected = false; + if (!task->hedge_started.compare_exchange_strong(expected, true)) + return false; // someone else is hedging (or already did) -> fall back to completion.wait() + + /// Cost cap: bound concurrent hedges so a slow region can't double all traffic. + if (hedges_inflight.fetch_add(1) >= hedged_read_max_inflight) + { + hedges_inflight.fetch_sub(1); + return false; + } + + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherHedgedReads); + bool ok = false; + try + { + /// Read the same range on this (already-blocked) consumer thread, racing the primary. + task->hedge_buf.resize(task->length); + readSync(task->hedge_buf.data(), task->length, task->offset); + task->hedge_winner.store(2, std::memory_order_release); + task->completion.notify(); // wake sharers waiting on the primary; they'll use hedge_buf + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherHedgedWins); + ok = true; + } + catch (...) + { + task->hedge_buf = {}; // hedge failed; fall back to the primary read + } + hedges_inflight.fetch_sub(1); + return ok; +} + std::span Prefetcher::getRangeData(const PrefetchHandle & request) { const RequestState * req = request.request; @@ -437,18 +743,41 @@ std::span Prefetcher::getRangeData(const PrefetchHandle & request) if (s == Task::State::Scheduled) { - s = runTask(task); + /// Inline read on this consumer (non-io_runner) thread: safe to split into parallel + /// segment reads submitted to io_runner. + s = runTask(task, /*allow_split*/ true); chassert(s != Task::State::Scheduled); } if (s == Task::State::Running) // (not `else`, the runTask above may return Running) { - task->completion.wait(); + /// Hedging: if the primary read hasn't finished within the threshold, issue a duplicate + /// read to cut the S3 GET tail. Eligible only for real (non-cached) remote reads no + /// larger than the size cap. hedgeReadSync waits up to the threshold itself. + bool served_by_hedge = false; + if (hedged_read_threshold_ms > 0 && read_mode == ReadMode::RandomRead + && !task->cached_region.has_value() && task->length > 0 + && (hedged_read_max_bytes == 0 || task->length <= hedged_read_max_bytes)) + { + served_by_hedge = hedgeReadSync(task); + } + + if (!served_by_hedge) + task->completion.wait(); s = task->state.load(); } ProfileEvents::increment(ProfileEvents::ParquetFetchWaitTimeMicroseconds, wait_time.elapsedMicroseconds()); } + + /// If a hedge produced the data, use it regardless of the primary's state (which may still be + /// Running, or even Exception if the primary failed but the hedge succeeded). + if (task->hedge_winner.load(std::memory_order_acquire) == 2) + { + chassert(req->task_offset + req->length <= task->hedge_buf.size()); + return std::span(task->hedge_buf.data() + req->task_offset, req->length); + } + if (s == Task::State::Exception) rethrowException(task); chassert(s == Task::State::Done); @@ -471,7 +800,7 @@ std::span Prefetcher::getRangeData(const PrefetchHandle & request) return std::span(task->buf.data() + req->task_offset, req->length); } -Prefetcher::Task::State Prefetcher::runTask(Task * task) +Prefetcher::Task::State Prefetcher::runTask(Task * task, bool allow_split) { auto s = Task::State::Scheduled; if (!task->state.compare_exchange_strong(s, Task::State::Running)) @@ -520,8 +849,35 @@ Prefetcher::Task::State Prefetcher::runTask(Task * task) else { task->buf.resize(task->length); - readSync(task->buf.data(), task->length, task->offset); + /// Split a boundary-straddling read into per-part segments fetched in parallel: a single + /// GET crossing a part boundary pays ~one extra RTT mid-stream, so aligned parallel reads + /// are faster. Only on the inline consumer path (allow_split) - deadlock-safe there. + std::vector pts; + if (allow_split && split_reads_across_boundaries && read_mode == ReadMode::RandomRead + && task->length >= (1ul << 20)) + pts = splitPointsForRange(task->offset, task->length); + + if (!pts.empty()) + { + std::vector> segs; + segs.reserve(pts.size() + 1); + size_t seg_start = task->offset; + for (size_t p : pts) + { + segs.emplace_back(task->buf.data() + (seg_start - task->offset), p - seg_start, seg_start); + seg_start = p; + } + segs.emplace_back(task->buf.data() + (seg_start - task->offset), task->offset + task->length - seg_start, seg_start); + readSyncParallel(segs); + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherSplitReadTasks); + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherSplitReadSegments, segs.size()); + } + else + { + readSync(task->buf.data(), task->length, task->offset); + } } + total_bytes_read.fetch_add(task->length, std::memory_order_relaxed); } catch (...) { @@ -540,6 +896,7 @@ Prefetcher::Task::State Prefetcher::runTask(Task * task) chassert(s == Task::State::Deallocated); task->buf = {}; task->cached_region.reset(); + task->hedge_buf = {}; } task->completion.notify(); diff --git a/src/Processors/Formats/Impl/Parquet/Prefetcher.h b/src/Processors/Formats/Impl/Parquet/Prefetcher.h index 40796dd10342..4309cb2b51a2 100644 --- a/src/Processors/Formats/Impl/Parquet/Prefetcher.h +++ b/src/Processors/Formats/Impl/Parquet/Prefetcher.h @@ -3,6 +3,8 @@ #include #include +#include +#include #include #include @@ -61,8 +63,31 @@ class Prefetcher /// Pass-through read from the underlying ReadBuffer. void readSync(char * to, size_t n, size_t offset); + /// Read several (to, n, offset) ranges concurrently: submits all but the first to io_runner and + /// runs the first inline, then waits for the rest. Falls back to sequential readSync when there's + /// no thread pool or a single range. The first exception (if any) is rethrown. + /// DEADLOCK SAFETY: must be called from a NON-io_runner thread (footer read at open, or the + /// inline consumer path in getRangeData). Calling it from a scheduled io_runner task could + /// deadlock (a pool thread waiting on pool tasks) - see ThreadPoolCallbackRunnerFast notes. + void readSyncParallel(const std::vector> & reads); + + /// Whether split_reads_across_boundaries is enabled (used by the footer read). + bool splitReadsEnabled() const { return split_reads_across_boundaries; } + + /// Retain a tail chunk of the file (the bytes already read to parse the footer) so that any + /// range subsequently registered and fully contained in it - notably the Column Index and + /// Offset Index, which live just below the FileMetaData footer - is served from this in-memory + /// copy instead of issuing another read. No-op for EntireFileIsInMemory (nothing to save) and + /// when the chunk is empty. `data` points at `length` bytes representing file offsets + /// [file_offset, file_offset + length). + void retainTail(const char * data, size_t length, size_t file_offset); + size_t getFileSize() const { return file_size; } + /// Average completed-read throughput (bytes/sec) since init, or 0 if not enough has been read to + /// estimate. Used for read back-pressure (input_format_parquet_prefetch_bandwidth_hide_seconds). + double averageThroughputBytesPerSec() const; + private: friend class PrefetchHandle; @@ -147,9 +172,18 @@ class Prefetcher std::atomic state {State::Scheduled}; /// How many RequestState-s in HasTask state point to this Task. std::atomic refcount {}; - /// Notified when the state changes from Running to Done or Exception. + /// Notified when the state changes from Running to Done or Exception (by the primary read, or + /// by a hedge read - whichever finishes first). CompletionNotification completion; std::exception_ptr exception; + + /// Hedging (tail-latency mitigation): a second read of the same range, raced against the + /// primary. Whichever finishes first CAS-wins `hedge_winner` (1 = primary, 2 = hedge) and + /// notifies `completion`; getRangeData then returns the winner's buffer. `hedge_buf` holds + /// the hedge read; `hedge_started` guards launching it at most once. + PaddedPODArray hedge_buf; + std::atomic hedge_winner {0}; + std::atomic hedge_started {false}; }; enum class ReadMode @@ -179,6 +213,42 @@ class Prefetcher size_t min_bytes_for_seek{}; size_t bytes_per_read_task{}; + /// Cumulative start offsets of the object's S3 multipart-upload parts, used to keep coalesced + /// read tasks within a single part. Empty = no alignment. Set once in init(), read-only after. + /// Takes precedence over read_alignment_stride when non-empty. + std::vector multipart_part_offsets; + + /// Fixed boundary grid (bytes) used for read alignment when multipart_part_offsets is empty. + /// 0 = off. And the anti-fragmentation min aligned-segment size. Set in init(), read-only after. + size_t read_alignment_stride = 0; + size_t read_alignment_min_bytes = 0; + + /// Split straddling reads into per-boundary segments read in parallel (see ReadOptions). Set in + /// init(), read-only after. + bool split_reads_across_boundaries = false; + + /// Anti-amplification: minimum "wanted / span" fraction for a coalesced task (see ReadOptions). + /// 0 = off. Set in init(), read-only after. + double read_min_fill_ratio = 0; + + /// Hedged reads (tail-latency mitigation). Set in init(), read-only after. hedges_inflight is the + /// live count, bounded by hedged_read_max_inflight. + size_t hedged_read_threshold_ms = 0; + size_t hedged_read_max_bytes = 0; + size_t hedged_read_max_inflight = 0; + std::atomic hedges_inflight {0}; + + /// Tail chunk retained by retainTail() to serve fully-contained ranges (Column/Offset Index) + /// without a second read. Written once before any prefetching, read-only afterwards. + /// [retained_tail_start, retained_tail_end) are file offsets; empty range == nothing retained. + PaddedPODArray retained_tail; + size_t retained_tail_start = 0; + size_t retained_tail_end = 0; + + /// Total bytes read by completed tasks, and when reading started, for throughput estimation. + std::atomic total_bytes_read{0}; + std::chrono::steady_clock::time_point read_start_time{}; + std::shared_ptr shutdown = std::make_shared(); /// Locked when creating a Task. @@ -202,8 +272,35 @@ class Prefetcher void pickRangesAndCreateTaskIfNotExists(RequestState *, const PrefetchHandle &, bool splitting, size_t start_offset, size_t end_offset, std::unique_lock lock); static void decreaseTaskRefcount(Task * task, size_t amount); void scheduleTask(Task * task); - Task::State runTask(Task * task); + /// allow_split: whether a boundary-straddling read may be split into parallel segments (segments + /// run on getIOThreadPool, a separate pool - deadlock-safe from any caller). Set true ONLY on the + /// inline getRangeData path, where a consumer is actively blocked (prefetch didn't cover this read + /// in time - i.e. ramp-up / prefetch behind). Prefetched (scheduled) reads pass false: no one is + /// waiting on them, so splitting would only add GETs. This makes splitting self-limiting - it + /// happens early / when starved and stops once prefetch keeps the pipeline full. + Task::State runTask(Task * task, bool allow_split = false); + /// File-offset split points strictly inside (offset, offset+length), from multipart_part_offsets + /// (preferred) or read_alignment_stride. Empty = the range fits within one part / no boundaries. + std::vector splitPointsForRange(size_t offset, size_t length) const; + /// Whether the byte range [lo, hi) contains a part boundary strictly inside - i.e. reading it + /// would straddle a part. Used to stop coalescing from bridging a gap across a boundary. + bool gapCrossesBoundary(size_t lo, size_t hi) const; + /// Anti-amplification: whether extending a task (currently covering [start_offset, end_offset) with + /// `wanted` bytes of actual ranges) to include range r would drop the task's wanted/span fraction + /// below read_min_fill_ratio (only for gaps above a small floor). Used to stop coalescing from + /// amplifying a tiny scattered column into a huge mostly-filler read. + bool fillRatioWouldBreak(size_t wanted, size_t start_offset, size_t end_offset, const RangeState & r) const; [[noreturn]] void rethrowException(Task * task); + + /// Issue a duplicate (hedged) read of `task`'s range when the primary is slow, to cut tail + /// latency. Phase A: the read runs synchronously on the calling (already-blocked) consumer + /// thread into task->hedge_buf; on success sets hedge_winner=2 and notifies so sharers wake. + /// Returns true if a usable hedge result was produced. At most once per task (hedge_started + /// guard), bounded by hedged_read_max_inflight, and only for real (non-cached) remote reads. + /// NOTE: a fully async race (Phase B) would need decreaseTaskRefcount to defer freeing + /// hedge_buf until an in-flight hedge finishes (the refcount/Deallocated path assumes one + /// reader) - deferred to avoid a use-after-free on hedge_buf. + bool hedgeReadSync(Task * task); }; /// Pins a pre-registered range that we may want to read. diff --git a/src/Processors/Formats/Impl/Parquet/ReadCommon.cpp b/src/Processors/Formats/Impl/Parquet/ReadCommon.cpp index 953562e818b4..beedef04a9e8 100644 --- a/src/Processors/Formats/Impl/Parquet/ReadCommon.cpp +++ b/src/Processors/Formats/Impl/Parquet/ReadCommon.cpp @@ -3,18 +3,22 @@ #include #include +#include + namespace DB::Parquet { -SharedResourcesExt::Limits SharedResourcesExt::getLimitsPerReader(const FormatParserSharedResources & parser_shared_resources, double fraction) +SharedResourcesExt::Limits SharedResourcesExt::getLimitsPerReader(const FormatParserSharedResources & parser_shared_resources, double memory_fraction, double thread_fraction) { const SharedResourcesExt & ext = *static_cast(parser_shared_resources.opaque.get()); size_t n = parser_shared_resources.num_streams.load(std::memory_order_relaxed); - fraction /= static_cast(std::max(n, size_t(1))); + /// Split each budget across the files read in parallel. + memory_fraction /= static_cast(std::max(n, size_t(1))); + thread_fraction /= static_cast(std::max(n, size_t(1))); return Limits { - .memory_low_watermark = size_t(ext.total_memory_low_watermark * fraction), - .memory_high_watermark = size_t(ext.total_memory_high_watermark * fraction), - .parsing_threads = size_t(std::max(std::lround(parser_shared_resources.parsing_runner.getMaxThreads() * fraction + .5), 1l))}; + .memory_low_watermark = size_t(ext.total_memory_low_watermark * memory_fraction), + .memory_high_watermark = size_t(ext.total_memory_high_watermark * memory_fraction), + .parsing_threads = size_t(std::max(std::lround(parser_shared_resources.parsing_runner.getMaxThreads() * thread_fraction + .5), 1l))}; } #ifdef OS_LINUX @@ -56,6 +60,35 @@ void CompletionNotification::notify() futexWake(&val, INT32_MAX); } +bool CompletionNotification::wait_for(UInt64 timeout_ms) +{ + UInt32 n = val.load(std::memory_order_acquire); + if (n == NOTIFIED) + return true; + if (n == EMPTY) + { + if (!val.compare_exchange_strong(n, WAITING)) + { + if (n == NOTIFIED) + return true; + chassert(n == WAITING); + } + } + const auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms); + while (true) + { + n = val.load(); + if (n == NOTIFIED) + return true; + chassert(n == WAITING); + const auto now = std::chrono::steady_clock::now(); + if (now >= deadline) + return false; + const UInt64 remaining_ns = std::chrono::duration_cast(deadline - now).count(); + futexTimedWait(&val, WAITING, remaining_ns); // may wake spuriously / on partial timeout; loop re-checks + } +} + #else bool CompletionNotification::check() const @@ -75,6 +108,13 @@ void CompletionNotification::notify() promise.set_value(); } +bool CompletionNotification::wait_for(UInt64 timeout_ms) +{ + if (check()) + return true; + return future.wait_for(std::chrono::milliseconds(timeout_ms)) == std::future_status::ready; +} + #endif } diff --git a/src/Processors/Formats/Impl/Parquet/ReadCommon.h b/src/Processors/Formats/Impl/Parquet/ReadCommon.h index 76b8a0fbccd5..038caaa345d4 100644 --- a/src/Processors/Formats/Impl/Parquet/ReadCommon.h +++ b/src/Processors/Formats/Impl/Parquet/ReadCommon.h @@ -3,6 +3,9 @@ #include #include +#include +#include + namespace DB { struct FormatParserSharedResources; @@ -36,8 +39,83 @@ struct ReadOptions /// probability becomes very high. E.g. if bloom filter has 1% false positive probability, /// searching for 100 elements would have 63% false positive probability. size_t bloom_filter_max_set_size = 100; + + /// Hint (in bytes) for how much of the file tail to read when fetching the parquet footer + /// (FileMetaData). 0 = unknown -> use the default speculative 64 KiB read. A data lake that + /// already knows per-file stats (e.g. Iceberg: column count, row-group count from split_offsets, + /// and per-column bound sizes) can set this via estimateParquetFooterSize so the footer is + /// captured in a single read for wide / many-row-group files. Only affects read count, never + /// correctness: an undershoot falls back to a second read, an overshoot reads a slightly larger + /// (already-clamped) tail. + size_t footer_metadata_size_hint = 0; + + /// Cumulative start offsets of the object's S3 multipart-upload parts (part i covers + /// [multipart_part_offsets[i], multipart_part_offsets[i+1])). Learned from GetObjectAttributes + /// (see ObjectStorageIdentityCache) and used to align coalesced read tasks to part boundaries so + /// a single read never straddles two parts. Empty = unknown / single-part -> no alignment. + /// EXPERIMENTAL: used to measure the effect of part-boundary-aligned reads. Takes precedence over + /// read_alignment_stride when non-empty. + std::vector multipart_part_offsets; + + /// Fixed boundary grid (bytes) for read alignment when the real per-file part layout is unknown: + /// no coalesced read straddles a multiple of this. 0 = off. EXPERIMENTAL. + size_t read_alignment_stride = 0; + + /// Anti-fragmentation guard: don't cut a read at an alignment boundary if the resulting aligned + /// segment would be smaller than this many bytes (allow the straddle instead of a tiny read). + size_t read_alignment_min_bytes = 0; + + /// Split a single read that straddles a part boundary into per-part segments read in parallel, + /// instead of one straddling GET. Measured on same-region AWS S3: a boundary-crossing ranged GET + /// pays ~one extra RTT mid-stream, so two aligned reads fetched concurrently are ~2.5x faster. + /// Boundaries come from multipart_part_offsets (preferred) or read_alignment_stride. Also enables + /// the speculative-parallel footer read (last 2 MiB + the rest, fired concurrently). 0 = off. + /// EXPERIMENTAL. Only helps latency-bound / critical-path reads; hidden by high concurrency. + bool split_reads_across_boundaries = false; + + /// Anti-amplification guard for coalescing: don't bridge a gap between two wanted ranges if the + /// resulting read task would be less than this fraction "wanted" (i.e. mostly filler bytes). + /// Coalescing normally reads through gaps shorter than min_bytes_for_seek to save a seek, but many + /// such sub-threshold gaps can accumulate into a task that is almost entirely unwanted bytes - + /// e.g. reading a tiny, scattered (RLE/near-constant) column drags in the neighbouring columns and + /// amplifies a few KB into hundreds of MB. This caps that: a task can never be more than + /// 1/read_min_fill_ratio times its wanted bytes. 0 = off (pure gap-size coalescing). Gaps below a + /// small absolute floor are always bridged regardless, so dense reads are unaffected. EXPERIMENTAL. + double read_min_fill_ratio = 0; + + /// Hedged reads (tail-latency mitigation): if a read a consumer is blocked on hasn't completed + /// within hedged_read_threshold_ms, issue a duplicate read and take whichever returns first. + /// 0 = off. Only reads no larger than hedged_read_max_bytes are hedged (latency, not throughput), + /// and at most hedged_read_max_inflight hedges run concurrently (cost cap). EXPERIMENTAL. + size_t hedged_read_threshold_ms = 0; + size_t hedged_read_max_bytes = 0; + size_t hedged_read_max_inflight = 0; }; +/// Estimate the serialized size of a parquet FileMetaData footer, to size the initial tail read. +/// - num_columns: number of leaf columns, +/// - num_row_groups: number of row groups, +/// - bounds_bytes: total bytes of the per-column min+max bounds for one file (e.g. summed from an +/// Iceberg manifest's lower_bounds + upper_bounds); these repeat per row group in the footer. +/// The result is clamped to a sane speculative-read range, so it is always safe to use directly as +/// ReadOptions::footer_metadata_size_hint. +inline size_t estimateParquetFooterSize(size_t num_columns, size_t num_row_groups, size_t bounds_bytes) +{ + /// Rough per-structure sizes of thrift-compact FileMetaData (see the constants' rationale in the + /// design notes). Overestimating only costs a marginally larger tail read; underestimating just + /// triggers the reader's existing second-read fallback. + constexpr size_t fixed_overhead = 4096; /// schema + file-level key/value metadata + constexpr size_t per_row_group = 64; /// RowGroupMetaData wrapper + constexpr size_t per_column_chunk = 112; /// ColumnMetaData fixed fields (offsets, sizes, ...) + constexpr size_t floor_size = 64ul << 10; /// never smaller than the default speculative read + constexpr size_t cap_size = 16ul << 20; /// don't speculatively read an enormous tail + + size_t est = fixed_overhead + + num_row_groups * (per_row_group + num_columns * per_column_chunk + bounds_bytes); + est += est / 3; /// ~1.3x safety for column names and Iceberg-vs-parquet truncation-length skew + return std::clamp(est, floor_size, cap_size); +} + struct SharedResourcesExt { size_t total_memory_low_watermark = 0; @@ -50,7 +128,7 @@ struct SharedResourcesExt size_t parsing_threads; }; - static Limits getLimitsPerReader(const FormatParserSharedResources & parser_shared_resources, double fraction); + static Limits getLimitsPerReader(const FormatParserSharedResources & parser_shared_resources, double memory_fraction, double thread_fraction); }; @@ -88,6 +166,10 @@ enum class ReadStage ColumnIndexAndOffsetIndex, OffsetIndex, + /// Issues the compressed data-page reads (startPrefetch) but does not decode. Charged to its own + /// memory budget so many row groups can have their reads in flight (deep prefetch) while only a + /// few are decoded at once (ColumnData). Decouples fetch depth from decode-ahead depth. + ColumnDataPrefetch, ColumnData, Deliver, @@ -186,6 +268,9 @@ class MemoryUsageToken val += amount; } + /// How much memory this token currently charges. + size_t charged() const { return val; } + private: ReadStage alloc_stage = ReadStage::Deallocated; size_t val = 0; @@ -209,6 +294,8 @@ class CompletionNotification public: bool check() const; void wait(); + /// Wait up to timeout_ms. Returns true if notified, false on timeout. + bool wait_for(UInt64 timeout_ms); void notify(); }; @@ -224,6 +311,8 @@ class CompletionNotification public: bool check() const; void wait(); + /// Wait up to timeout_ms. Returns true if notified, false on timeout. + bool wait_for(UInt64 timeout_ms); void notify(); }; diff --git a/src/Processors/Formats/Impl/Parquet/ReadManager.cpp b/src/Processors/Formats/Impl/Parquet/ReadManager.cpp index 3408cd99c032..ae61ef8084a4 100644 --- a/src/Processors/Formats/Impl/Parquet/ReadManager.cpp +++ b/src/Processors/Formats/Impl/Parquet/ReadManager.cpp @@ -53,7 +53,7 @@ void ReadManager::init(FormatParserSharedResourcesPtr parser_shared_resources_, parser_shared_resources = parser_shared_resources_; if (reader.file_metadata.schema.empty()) - reader.file_metadata = Reader::readFileMetaData(reader.prefetcher); + reader.file_metadata = Reader::readFileMetaData(reader.prefetcher, reader.options.footer_metadata_size_hint); if (buckets_to_read_) { @@ -74,17 +74,58 @@ void ReadManager::init(FormatParserSharedResourcesPtr parser_shared_resources_, stages[i].row_group_tasks_to_schedule.resize(num_row_groups); } - /// Distribute memory budget among stages. - /// The distribution is static to make sure no stage gets starved if others eat all the memory. - /// E.g. if the budget was shared among all stages, maybe PrewhereData could run far ahead and - /// The distribution is static to make sure no stage gets starved if others eat all the memory. - double sum = 0; - stages[size_t(ReadStage::NotStarted)].memory_target_fraction = 0; - stages[size_t(ReadStage::Deliver)].memory_target_fraction = 0; + /// Distribute the memory and thread budgets among stages. + /// The distribution is static to make sure no stage gets starved if others eat all the resources. + /// E.g. if the budget was shared among all stages, maybe ColumnData could run far ahead and eat + /// all the memory, starving the small index reads that other row groups need to make progress. + /// + /// Memory and threads are budgeted separately (memory_target_fraction vs thread_target_fraction): + /// the main data (ColumnData) needs most of the memory because decoded row groups are large, but + /// the small index/bloom reads are latency-bound over network and want lots of parallelism while + /// costing little memory. A single shared fraction (the previous behavior, all stages equal at + /// 0.2) forced trading one for the other: ColumnData was capped at 0.2 of the budget AND 0.2 of + /// the threads, so on a single large remote file only ~2 row groups were read/decoded ahead, + /// leaving the link idle. ColumnData now gets the lion's share of memory and a larger thread + /// share (so more row groups are in flight, keeping more reads outstanding), while the index + /// stages keep enough threads to issue their many small reads in parallel. + /// + /// These are static starting points; they still couple prefetch depth to decode concurrency (a + /// ColumnData slot both issues the reads and holds the decoded row group). Decoupling those - so + /// many compressed row groups can be in flight without as many decoded ones resident - is a + /// separate, larger change; see the design notes. The numbers below want a perf run to tune. + using S = ReadStage; + auto set_fractions = [&](S s, double memory_fraction, double thread_fraction) + { + stages[size_t(s)].memory_target_fraction = memory_fraction; + stages[size_t(s)].thread_target_fraction = thread_fraction; + }; + set_fractions(S::NotStarted, 0, 0); + set_fractions(S::BloomFilterHeader, 0.05, 1); + set_fractions(S::BloomFilterBlocksOrDictionary, 0.10, 1); + set_fractions(S::ColumnIndexAndOffsetIndex, 0.05, 1); + set_fractions(S::OffsetIndex, 0.05, 1); + /// Compressed data-page reads: a large budget so many row groups can prefetch ahead (each + /// compressed row group is cheap). Few threads - issuing startPrefetch is cheap and the actual + /// reads run in the Prefetcher's own io pool, not the parsing threads. + set_fractions(S::ColumnDataPrefetch, 0.45, 1); + /// Decode: the expensive resource (decoded row groups are large), so a bounded memory budget and + /// the bulk of the decode threads. This bounds how many row groups are decoded/resident at once, + /// independently of how deep the compressed prefetch runs above. + set_fractions(S::ColumnData, 0.30, 3); + set_fractions(S::Deliver, 0, 0); + + double memory_sum = 0; + double thread_sum = 0; for (const Stage & stage : stages) - sum += stage.memory_target_fraction; + { + memory_sum += stage.memory_target_fraction; + thread_sum += stage.thread_target_fraction; + } for (Stage & stage : stages) - stage.memory_target_fraction /= sum; + { + stage.memory_target_fraction /= memory_sum; + stage.thread_target_fraction /= thread_sum; + } /// The NotStarted stage completed for all row groups, transition to next stage. MemoryUsageDiff diff(ReadStage::NotStarted); @@ -139,6 +180,7 @@ void ReadManager::finishRowGroupStage(size_t row_group_idx, ReadStage stage, Mem switch (stage) { case ReadStage::NotStarted: + case ReadStage::ColumnDataPrefetch: case ReadStage::ColumnData: case ReadStage::Deliver: chassert(false); @@ -295,9 +337,10 @@ void ReadManager::addTasksToReadColumns(size_t row_group_idx, size_t row_subgrou } else { - LOG_TEST(getLogger("ParquetReadManager"), "addTasksToReadColumns: added ColumnData: i={} step_idx={} row_group_idx={} row_subgroup_idx={}", i, step_idx, row_group_idx, row_subgroup_idx); + /// `stage` is ColumnDataPrefetch (issue reads) or ColumnData (decode). + LOG_TEST(getLogger("ParquetReadManager"), "addTasksToReadColumns: added {}: i={} step_idx={} row_group_idx={} row_subgroup_idx={}", magic_enum::enum_name(stage), i, step_idx, row_group_idx, row_subgroup_idx); add_tasks.push_back(Task { - .stage = ReadStage::ColumnData, + .stage = stage, .step_idx = step_idx, .row_group_idx = row_group_idx, .row_subgroup_idx = row_subgroup_idx, @@ -307,8 +350,8 @@ void ReadManager::addTasksToReadColumns(size_t row_group_idx, size_t row_subgrou if (add_tasks.empty() && is_offset_index) { - /// Don't need to read offset index, move on to next stage (ColumnData). - stage = ReadStage::ColumnData; + /// Don't need to read offset index, move on to the next stage (ColumnDataPrefetch). + stage = ReadStage::ColumnDataPrefetch; continue; } @@ -319,7 +362,7 @@ void ReadManager::addTasksToReadColumns(size_t row_group_idx, size_t row_subgrou /// (RowSubgroup.filter.memory) work correctly when PREWHERE expression doesn't use any /// columns (note: the expression may still be nontrivial, e.g. `rand()%2=0`).) add_tasks.push_back(Task { - .stage = ReadStage::ColumnData, + .stage = stage, .step_idx = step_idx, .row_group_idx = row_group_idx, .row_subgroup_idx = row_subgroup_idx, @@ -420,6 +463,13 @@ void ReadManager::finishRowSubgroupStage(size_t row_group_idx, size_t row_subgro case ReadStage::ColumnIndexAndOffsetIndex: case ReadStage::OffsetIndex: { + /// Prerequisites read; issue the compressed data-page reads (but don't decode yet). + addTasksToReadColumns(row_group_idx, row_subgroup_idx, ReadStage::ColumnDataPrefetch, step_idx, diff); + return; + } + case ReadStage::ColumnDataPrefetch: + { + /// Data-page reads issued (in flight in the Prefetcher's io pool); now decode. addTasksToReadColumns(row_group_idx, row_subgroup_idx, ReadStage::ColumnData, step_idx, diff); return; } @@ -544,7 +594,7 @@ void ReadManager::flushMemoryUsageDiff(MemoryUsageDiff && diff) if (!should_schedule && d < 0) { const auto & stage = stages[i]; - auto limits = SharedResourcesExt::getLimitsPerReader(*parser_shared_resources, stage.memory_target_fraction); + auto limits = SharedResourcesExt::getLimitsPerReader(*parser_shared_resources, stage.memory_target_fraction, stage.thread_target_fraction); should_schedule = checkTaskSchedulingLimits( stage.memory_usage.load(std::memory_order_relaxed), 0, stage.batches_in_progress.load(std::memory_order_relaxed), 0, limits); @@ -562,10 +612,24 @@ void ReadManager::scheduleTasksIfNeeded(ReadStage stage_idx) MemoryUsageDiff diff(stage_idx); std::vector tasks; - auto limits = SharedResourcesExt::getLimitsPerReader(*parser_shared_resources, stage.memory_target_fraction); + auto limits = SharedResourcesExt::getLimitsPerReader(*parser_shared_resources, stage.memory_target_fraction, stage.thread_target_fraction); size_t memory_usage = stage.memory_usage.load(std::memory_order_relaxed); size_t batches_in_progress = stage.batches_in_progress.load(std::memory_order_relaxed); + /// Read back-pressure (finding #4): once the compressed data already in flight covers more than + /// `prefetch_bandwidth_hide_seconds` of the measured read throughput, the storage link is fed, so + /// stop prefetching further ahead - extra compressed buffering would only waste memory without + /// improving throughput. The compressed in-flight bytes are exactly this stage's memory usage. + /// Fail-open: 0 (disabled), or unknown throughput, means no throttle. The privileged-task escape + /// below still applies, so this can never deadlock. + double prefetch_ahead_bytes_limit = 0; // 0 = no back-pressure + if (stage_idx == ReadStage::ColumnDataPrefetch && reader.options.format.parquet.prefetch_bandwidth_hide_seconds > 0) + { + double throughput = reader.prefetcher.averageThroughputBytesPerSec(); + if (throughput > 0) + prefetch_ahead_bytes_limit = throughput * reader.options.format.parquet.prefetch_bandwidth_hide_seconds; + } + LOG_TEST(getLogger("ParquetReadManager"), "scheduleTasksIfNeeded: stage={} memory_usage={} batches_in_progress={} limits: mem_low={} mem_high={} threads={}", magic_enum::enum_name(stage_idx), memory_usage, batches_in_progress, limits.memory_low_watermark, limits.memory_high_watermark, limits.parsing_threads); @@ -599,6 +663,14 @@ void ReadManager::scheduleTasksIfNeeded(ReadStage stage_idx) bool can_schedule = checkTaskSchedulingLimits( memory_usage, size_t(diff.by_stage[size_t(stage_idx)]), batches_in_progress, tasks.size(), limits); + /// Bandwidth back-pressure: hold off further prefetch when the link is already fed. + if (can_schedule && prefetch_ahead_bytes_limit > 0) + { + double compressed_in_flight = static_cast(memory_usage) + + static_cast(std::max(0, diff.by_stage[size_t(stage_idx)])); + if (compressed_in_flight >= prefetch_ahead_bytes_limit) + can_schedule = false; + } bool is_privileged = is_privileged_task(row_group_idx); LOG_TEST(getLogger("ParquetReadManager"), "scheduleTasksIfNeeded: stage={} rg={} can_schedule={} is_privileged={}", magic_enum::enum_name(stage_idx), row_group_idx, can_schedule, is_privileged); @@ -715,13 +787,17 @@ void ReadManager::scheduleTask(Task task, bool is_first_in_group, MemoryUsageDif case ReadStage::OffsetIndex: prefetches.push_back(&column.offset_index_prefetch); break; - case ReadStage::ColumnData: + case ReadStage::ColumnDataPrefetch: { RowSubgroup & row_subgroup = row_group.subgroups.at(task.row_subgroup_idx); - ColumnSubchunk & subchunk = row_subgroup.columns.at(task.column_idx); if (row_subgroup.filter.rows_pass == 0) break; - reader.determinePagesToPrefetch(column, row_subgroup, row_group, prefetches); + /// Determine which data pages this subgroup needs and queue their reads. The + /// startPrefetch at the end of this function issues them against the Prefetcher's io + /// pool and charges the compressed bytes to the ColumnDataPrefetch stage budget - + /// separate from the decoded-output budget (ColumnData) - so many row groups can have + /// their reads in flight (deep prefetch) while only a few are decoded at once. + reader.determinePagesToPrefetch(column, row_subgroup.columns.at(task.column_idx), reader.primitive_columns.at(task.column_idx), row_subgroup, row_group, prefetches); /// Side note: would be nice to avoid reading the dictionary if all dictionary-encoded /// pages were filtered out (e.g. if it's a 100 MB column chunk with unique long strings, @@ -737,7 +813,17 @@ void ReadManager::scheduleTask(Task task, bool is_first_in_group, MemoryUsageDif { prefetches.push_back(&column.data_pages_prefetch); } - + break; + } + case ReadStage::ColumnData: + { + RowSubgroup & row_subgroup = row_group.subgroups.at(task.row_subgroup_idx); + ColumnSubchunk & subchunk = row_subgroup.columns.at(task.column_idx); + if (row_subgroup.filter.rows_pass == 0) + break; + /// The data-page reads were already issued in ColumnDataPrefetch (and are in flight or + /// done in the Prefetcher). Here we only reserve the estimated decoded-output memory + /// against the ColumnData budget; runTask then decodes from those buffers. double bytes_per_row = reader.estimateColumnMemoryBytesPerRow(column, row_group, reader.primitive_columns.at(task.column_idx)); size_t column_memory = static_cast(bytes_per_row * static_cast(row_subgroup.filter.rows_pass)); subchunk.column_and_offsets_memory = MemoryUsageToken(column_memory, &diff); @@ -842,6 +928,11 @@ void ReadManager::runTask(Task task, bool last_in_batch, MemoryUsageDiff & diff) reader.decodeOffsetIndex(column, row_group); column.offset_index_prefetch.reset(&diff); break; + case ReadStage::ColumnDataPrefetch: + /// The compressed data-page reads were already issued in scheduleTask (startPrefetch) + /// and proceed asynchronously in the Prefetcher's io pool. Nothing to do here; the + /// subgroup advances to ColumnData, which decodes from those buffers. + break; case ReadStage::ColumnData: { RowSubgroup & row_subgroup = row_group.subgroups.at(task.row_subgroup_idx); @@ -859,7 +950,7 @@ void ReadManager::runTask(Task task, bool last_in_batch, MemoryUsageDiff & diff) chassert(task.row_subgroup_idx != UINT64_MAX); reader.decodePrimitiveColumn( column, column_info, row_subgroup.columns.at(task.column_idx), - row_group, row_subgroup); + row_group, row_subgroup, diff); for (size_t i = prev_page_idx; i < column.data_pages_idx; ++i) { diff --git a/src/Processors/Formats/Impl/Parquet/ReadManager.h b/src/Processors/Formats/Impl/Parquet/ReadManager.h index 49ac1b2942c8..e9f71d4aeb2f 100644 --- a/src/Processors/Formats/Impl/Parquet/ReadManager.h +++ b/src/Processors/Formats/Impl/Parquet/ReadManager.h @@ -88,7 +88,14 @@ class ReadManager /// Tasks that are either in thread pool's queue or executing. std::atomic batches_in_progress {0}; + /// Share of the query-global memory budget for this stage. Kept separate from the thread + /// share below so a stage that needs many parallel reads but little memory (e.g. the small + /// index reads, which are latency-bound over network) is not forced to trade one for the + /// other. See ReadManager's constructor for how these are assigned and normalized. double memory_target_fraction = 1; + /// Share of the parsing thread pool for this stage (parallelism), independent of the memory + /// share above. + double thread_target_fraction = 1; /// We take advantage of the fact that each pair can have at most one group /// of tasks in flight at a time. E.g. we create tasks to read columns in subgroup n, then diff --git a/src/Processors/Formats/Impl/Parquet/Reader.cpp b/src/Processors/Formats/Impl/Parquet/Reader.cpp index d94b6ae42b81..5b3421871868 100644 --- a/src/Processors/Formats/Impl/Parquet/Reader.cpp +++ b/src/Processors/Formats/Impl/Parquet/Reader.cpp @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -40,6 +41,9 @@ namespace ProfileEvents { extern const Event ParquetRowsFilterExpression; extern const Event ParquetColumnsFilterExpression; + extern const Event ParquetConstantColumnChunks; + extern const Event ParquetConstantColumnSubchunks; + extern const Event ParquetPrefetcherFooterSpeculativeParallel; } namespace DB::Parquet @@ -189,7 +193,7 @@ void Reader::init(const ReadOptions & options_, const Block & sample_block_, For format_filter_info = format_filter_info_; } -parq::FileMetaData Reader::readFileMetaData(Prefetcher & prefetcher) +parq::FileMetaData Reader::readFileMetaData(Prefetcher & prefetcher, size_t footer_size_hint) { /// Parquet file ends with: /// * serialized FileMetaData struct, @@ -200,15 +204,43 @@ parq::FileMetaData Reader::readFileMetaData(Prefetcher & prefetcher) if (file_size <= 8) throw Exception(ErrorCodes::INCORRECT_DATA, "Parquet file too short: {} bytes", file_size); - /// Read the last 64 KiB in hopes that FileMetaData is smaller than that. - /// This is usually enough for files smaller than a few hundred MB. - size_t initial_read_size = std::min(file_size, 64ul << 10); + /// Read a tail chunk in hopes that FileMetaData is smaller than that. Default 64 KiB is usually + /// enough for files smaller than a few hundred MB; a caller that knows more (e.g. Iceberg, via + /// footer_size_hint) can size it so wide / many-row-group files are captured in one read. If the + /// footer turns out larger, the code below reads the exact remainder (one extra read). + size_t initial_read_size = std::min(file_size, footer_size_hint ? footer_size_hint : (64ul << 10)); PODArray buf(initial_read_size); - prefetcher.readSync(buf.data(), initial_read_size, file_size - initial_read_size); + /// Speculative-parallel footer read: when the hint is large (> 2 MiB) we don't know if the footer + /// really needs the whole hinted tail, but tail reads up to ~2 MiB are latency-flat while a single + /// big read is bandwidth-bound. So fire two concurrent reads - the last 2 MiB (always holds the + /// footer length + magic, and the entire footer for small footers) and the preceding rest of the + /// hinted tail. This keeps the second request already in flight if the footer overflows 2 MiB, + /// avoiding a sequential extra round-trip; the buffer layout is identical to the single read. + static constexpr size_t footer_spec_tail = 2ul << 20; + if (prefetcher.splitReadsEnabled() && initial_read_size > footer_spec_tail) + { + size_t rest = initial_read_size - footer_spec_tail; + prefetcher.readSyncParallel({ + std::make_tuple(buf.data(), rest, file_size - initial_read_size), + std::make_tuple(buf.data() + rest, footer_spec_tail, file_size - footer_spec_tail), + }); + ProfileEvents::increment(ProfileEvents::ParquetPrefetcherFooterSpeculativeParallel); + } + else + { + prefetcher.readSync(buf.data(), initial_read_size, file_size - initial_read_size); + } if (memcmp(buf.data() + initial_read_size - 4, "PAR1", 4) != 0) throw Exception(ErrorCodes::INCORRECT_DATA, "Not a Parquet file (wrong magic bytes at the end of file)"); + /// The tail we just read also covers the Column Index / Offset Index (they sit just below the + /// FileMetaData footer). Retain it so those ranges are served from memory rather than re-read. + /// Retain the original tail bytes now, before the (rare) oversized-footer branch below mutates + /// `buf`. Especially effective together with footer_size_hint, which widens this read to span + /// the whole footer + index region of wide / many-row-group files. + prefetcher.retainTail(buf.data(), initial_read_size, file_size - initial_read_size); + int32_t metadata_size_i32 = 0; memcpy(&metadata_size_i32, buf.data() + initial_read_size - 8, 4); if (metadata_size_i32 <= 0 || size_t(metadata_size_i32) + 8 > file_size) @@ -422,6 +454,8 @@ void Reader::prefilterAndInitRowGroups(const std::optionalmeta_data.statistics.__isset.null_count && column.meta->meta_data.statistics.null_count == 0; column.need_null_map = is_nullable && !null_count_is_known_to_be_zero; + + detectConstantColumn(column, primitive_columns[column_idx]); } } @@ -544,7 +578,9 @@ void Reader::initializePrefetches() /// Dictionary page. size_t dict_page_length = 0; - if (column.meta->meta_data.__isset.dictionary_page_offset) + /// A constant column chunk is materialized without reading any pages (data or + /// dictionary), so don't prefetch its dictionary page either. + if (column.meta->meta_data.__isset.dictionary_page_offset && !column.is_constant) { /// We assume that the dictionary page is immediately followed by the first data page. size_t start = size_t(column.meta->meta_data.dictionary_page_offset); @@ -592,8 +628,23 @@ void Reader::initializePrefetches() max_header_length, /*likely_to_be_used=*/ true); } + /// A constant column chunk is materialized without reading any of its pages (see + /// detectConstantColumn and decodePrimitiveColumn), so it needs neither the offset index + /// nor the column index nor the data pages. The row group already passed the key + /// condition via its min == max hyperrectangle, so page-level pruning is redundant here. + + /// Force-load the column index (per-page stats) for an eligible column even without a + /// predicate on it, so tier-2 constant detection (detectConstantSubchunk) can run. Gated + /// by input_format_parquet_use_column_index_for_constant_columns. Requires the offset + /// index too (page -> row mapping), loaded just below. + const bool force_column_index_for_const = + options.format.parquet.use_column_index_for_constant_columns + && !column.is_constant + && constColumnMaterializationEligible(primitive_columns[column_idx]) + && column.meta->__isset.column_index_offset && column.meta->__isset.column_index_length; + /// Offset index. - if (use_offset_index && + if ((use_offset_index || force_column_index_for_const) && !column.is_constant && column.meta->__isset.offset_index_offset && column.meta->__isset.offset_index_length) { column.offset_index_prefetch = prefetcher.registerRange( @@ -602,7 +653,8 @@ void Reader::initializePrefetches() } /// Column index. - column.use_column_index = primitive_columns[column_idx].column_index_condition + column.use_column_index = !column.is_constant + && (primitive_columns[column_idx].column_index_condition || force_column_index_for_const) && column.offset_index_prefetch && column.meta->__isset.column_index_offset && column.meta->__isset.column_index_length; if (column.use_column_index) @@ -622,10 +674,11 @@ void Reader::initializePrefetches() if (file_metadata.created_by == "parquet-mr" && !column.meta->meta_data.__isset.dictionary_page_offset && !column.meta->__isset.offset_index_offset) data_pages_extra_bytes = std::min(100ul, prefetcher.getFileSize() - size_t(column.meta->meta_data.data_page_offset) - column.data_pages_bytes); - column.data_pages_prefetch = prefetcher.registerRange( - size_t(column.meta->meta_data.data_page_offset), - column.data_pages_bytes + data_pages_extra_bytes, - /*likely_to_be_used=*/ true); + if (!column.is_constant) + column.data_pages_prefetch = prefetcher.registerRange( + size_t(column.meta->meta_data.data_page_offset), + column.data_pages_bytes + data_pages_extra_bytes, + /*likely_to_be_used=*/ true); } } @@ -943,7 +996,9 @@ void Reader::applyColumnIndex(ColumnChunk & column, const PrimitiveColumnInfo & try { chassert(column.use_column_index); - chassert(column_info.column_index_condition); + /// column_index_condition may be null when the column index was force-loaded only for tier-2 + /// constant detection (input_format_parquet_use_column_index_for_constant_columns); in that + /// case we record per-page constant info below but do no page-level predicate pruning. auto data = prefetcher.getRangeData(column.column_index_prefetch); parq::ColumnIndex column_index; @@ -957,6 +1012,17 @@ void Reader::applyColumnIndex(ColumnChunk & column, const PrimitiveColumnInfo & (column_index.__isset.null_counts && column_index.null_counts.size() != num_pages)) throw Exception(ErrorCodes::INCORRECT_DATA, "Unexpected number of pages: {} null_pages, {} null_counts, {} min_values, {} max_values, {} pages in offset index", column_index.null_pages.size(), column_index.null_counts.size(), column_index.min_values.size(), column_index.max_values.size(), num_pages); + /// Tier 2 constant-column detection: retain per-page constant info from the Column Index for + /// use by detectConstantSubchunk. Only when eligible and the whole chunk is not already + /// constant (tier 1). No truncation guard is needed even though the Column Index has no + /// per-page exactness flag: bounds are always valid (min_value <= every value <= max_value) + /// and truncation only widens them, so a per-page min_value == max_value proves a single exact + /// value - for BYTE_ARRAY strings too (a truncated value would give min_value < max_value). + const bool record_page_const = + constColumnMaterializationEligible(column_info) && !column.is_constant; + if (record_page_const) + column.page_const_info.assign(num_pages, ColumnChunk::PageConstInfo{}); + Hyperrectangle hyperrectangle(extended_sample_block.columns(), Range::createWholeUniverse()); size_t prev_row_idx = 0; // start of the latest range of rows that pass filter for (size_t page_idx = 0; page_idx < num_pages; ++page_idx) @@ -965,7 +1031,15 @@ void Reader::applyColumnIndex(ColumnChunk & column, const PrimitiveColumnInfo & range = Range::createWholeUniverse(); bool always_null = !column_index.null_pages.empty() && column_index.null_pages[page_idx]; - bool can_be_null = !column_index.__isset.null_counts || column_index.null_counts[page_idx] != 0; + /// A required column (max def level 0) can never contain nulls, regardless of whether the + /// Column Index carries null_counts. This matters for tier-2 constant detection below: + /// ClickHouse's own writer only emits null_counts for nullable columns (see Write.cpp, + /// has_null_count = max_def == 1), so without the `nullable` guard can_be_null would be + /// spuriously true for every required column and the is_const branch would never fire. + bool can_be_null = nullable && (!column_index.__isset.null_counts || column_index.null_counts[page_idx] != 0); + + if (record_page_const) + column.page_const_info[page_idx].all_null = always_null; if (nullable && always_null) { @@ -980,20 +1054,36 @@ void Reader::applyColumnIndex(ColumnChunk & column, const PrimitiveColumnInfo & column_info.decoder.decodeField(column_index.min_values[page_idx], /*is_max=*/ false, range.left); column_info.decoder.decodeField(column_index.max_values[page_idx], /*is_max=*/ true, range.right); + /// A page with no nulls whose decoded min == max holds a single value (post-cast + /// output domain, like tier 1). Captured before adjustRangeFromIndexIfNeeded, which + /// mutates the range for null/default handling. + if (record_page_const && !can_be_null + && !range.left.isNull() && range.left == range.right) + { + column.page_const_info[page_idx].is_const = true; + column.page_const_info[page_idx].value = range.left; + } + adjustRangeFromIndexIfNeeded(range, column_info, can_be_null); } - bool passes_filter = column_info.column_index_condition->checkInHyperrectangle( - hyperrectangle, extended_sample_block_data_types).can_be_true; - - if (!passes_filter) + /// Page-level predicate pruning only when this column has a condition. When the index was + /// force-loaded solely for constant detection, prev_row_idx stays 0 and the whole chunk is + /// selected (the final range below), i.e. no restriction. + if (column_info.column_index_condition) { - size_t start_row = column.offset_index.page_locations[page_idx].first_row_index; - size_t end_row = page_idx + 1 < num_pages ? column.offset_index.page_locations[page_idx + 1].first_row_index : row_group.meta->num_rows; - chassert(end_row > start_row); // validated in decodeOffsetIndex - if (start_row > prev_row_idx) - column.row_ranges_after_column_index.emplace_back(prev_row_idx, start_row); - prev_row_idx = end_row; + bool passes_filter = column_info.column_index_condition->checkInHyperrectangle( + hyperrectangle, extended_sample_block_data_types).can_be_true; + + if (!passes_filter) + { + size_t start_row = column.offset_index.page_locations[page_idx].first_row_index; + size_t end_row = page_idx + 1 < num_pages ? column.offset_index.page_locations[page_idx + 1].first_row_index : row_group.meta->num_rows; + chassert(end_row > start_row); // validated in decodeOffsetIndex + if (start_row > prev_row_idx) + column.row_ranges_after_column_index.emplace_back(prev_row_idx, start_row); + prev_row_idx = end_row; + } } } @@ -1149,6 +1239,34 @@ void Reader::intersectColumnIndexResultsAndInitSubgroups(RowGroup & row_group) } if (options.format.defaults_for_omitted_fields) row_subgroup.block_missing_values.init(sample_block->columns()); + + /// Tier 2 constant detection: for each primitive column not already whole-chunk-constant + /// (tier 1), check whether it is constant over this subgroup's row range using the + /// per-page Column Index info retained in applyColumnIndex. If so, mark the subchunk so + /// decodePrimitiveColumn skips decoding it and formOutputColumn materializes a ColumnConst. + /// The subgroup's pages are still skipped forward by the next subgroup's + /// skipToRowOrNextPage; skipping the prefetch itself is a separate follow-up. + if (row_group.need_to_process) + { + for (size_t i = 0; i < primitive_columns.size(); ++i) + { + ColumnChunk & column = row_group.columns[i]; + if (column.is_constant) + continue; // tier 1 already materializes the whole chunk + bool all_null = false; + Field value; + if (detectConstantSubchunk( + column, primitive_columns[i], substart, subend, + size_t(row_group.meta->num_rows), all_null, value)) + { + ColumnSubchunk & subchunk = row_subgroup.columns[i]; + subchunk.is_constant = true; + subchunk.is_all_null = all_null; + subchunk.constant_value = std::move(value); + ProfileEvents::increment(ProfileEvents::ParquetConstantColumnSubchunks); + } + } + } } } row_group.intersected_row_ranges_after_column_index = std::move(row_ranges); @@ -1187,12 +1305,20 @@ void Reader::decodeOffsetIndex(ColumnChunk & column, const RowGroup & row_group) } } -void Reader::determinePagesToPrefetch(ColumnChunk & column, const RowSubgroup & row_subgroup, const RowGroup & row_group, std::vector & out) +void Reader::determinePagesToPrefetch(ColumnChunk & column, const ColumnSubchunk & subchunk, const PrimitiveColumnInfo & column_info, const RowSubgroup & row_subgroup, const RowGroup & row_group, std::vector & out) { chassert(row_subgroup.filter.rows_pass > 0); + if (column.is_constant) + return; // constant column: data pages are never read if (column.offset_index.page_locations.empty()) return; // no offset index, can't prefetch individual pages + /// If this subgroup will fill its single-value pages from the Column Index instead of decoding + /// them (input_format_parquet_fill_constant_pages), don't prefetch those pages. Deterministic, so + /// the decode path (willFillConstantPages) makes the same decision. A const page shared with a + /// subgroup that decodes it normally is still claimed by that subgroup and thus fetched. + const bool fill_const_pages = willFillConstantPages(column, column_info, row_group, row_subgroup); + if (column.data_pages.empty()) { const auto & locations = column.offset_index.page_locations; @@ -1254,6 +1380,25 @@ void Reader::determinePagesToPrefetch(ColumnChunk & column, const RowSubgroup & if (passes_filter && row_subgroup.filter.rows_pass < row_subgroup.filter.rows_total) passes_filter = !memoryIsZero(row_subgroup.filter.filter.data(), start_row_idx - row_subgroup.start_row_idx, end_row_idx - row_subgroup.start_row_idx); + /// Tier 2: this subgroup's column is constant (detectConstantSubchunk), so it decodes nothing + /// and needs no pages. Treat every page as not-needed-by-this-subgroup; a page fully inside + /// this subgroup is then released below (never fetched), while a page shared with a + /// neighbouring non-constant subgroup is left for that subgroup to claim. Safe because a + /// constant subgroup never calls skipToRowOrNextPage, and non-constant subgroups jump + /// directly to their own (claimed) pages via the offset index. + if (subchunk.is_constant) + passes_filter = false; + + /// Mixed-topology fill: this subgroup fills its single-value pages instead of decoding them, + /// so it does not need those pages. (Varying pages in the same subgroup are still claimed.) + if (passes_filter && fill_const_pages) + { + const size_t gid = size_t(page.meta - column.offset_index.page_locations.data()); + if (gid < column.page_const_info.size() + && (column.page_const_info[gid].is_const || column.page_const_info[gid].all_null)) + passes_filter = false; + } + if (passes_filter) out.push_back(&page.prefetch); // this subgroup needs this page else if (page.end_row_idx > subgroup_end) @@ -1327,8 +1472,341 @@ double Reader::estimateColumnMemoryBytesPerRow(const ColumnChunk & column, const return res; } -void Reader::decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info, ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup) +bool Reader::constColumnMaterializationEligible(const PrimitiveColumnInfo & column_info) const +{ + if (!options.format.parquet.use_constant_column_optimization) + return false; + /// We rely on min/max statistics being both present and decodable. + if (!column_info.decoder.allow_stats) + return false; + + /// Only flat, top-level primitive columns, so that one parquet value maps 1:1 to one output row + /// and formOutputColumn can materialize the value directly. Exclude: + /// - arrays (leaf repetition level > 0, or any array level: max_array_def > 0), + /// - leaves nested inside a Tuple/Map/Array output column (the output column is not primitive; + /// this also covers physically-nullable structs, whose leaves are not primitive outputs). + /// A plain Nullable(T) is fine: it adds a definition level but no repetition, and its output + /// column is still primitive; the no-nulls check and the output_nullable wrap handle it. + if (column_info.levels.back().rep != 0 || column_info.max_array_def != 0) + return false; + if (column_info.idx_in_output_block >= sample_block_to_output_columns_idx.size()) + return false; + const auto & output_idx = sample_block_to_output_columns_idx.at(column_info.idx_in_output_block); + if (!output_idx.has_value() || !output_columns[output_idx.value()].is_primitive) + return false; + return true; +} + +void Reader::detectConstantColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info) const +{ + if (!constColumnMaterializationEligible(column_info)) + return; + + const auto & meta_data = column.meta->meta_data; + if (!meta_data.__isset.statistics) + return; + const auto & stats = meta_data.statistics; + + bool physically_nullable = column_info.levels.back().def > 0; + + /// All-null chunk: every row is null (null_count == num_values). This is provable only for a + /// physically nullable leaf (a REQUIRED column can have no nulls), and only when the writer + /// emitted the null_count statistic. No value is decoded at all, so this skips the min/max + /// exactness and truncation concerns below. We materialize `Null` for a Nullable output, or the + /// output default when `null_as_default` substitutes nulls for a non-nullable output; a + /// non-nullable output without null substitution cannot represent the result, so we leave the + /// chunk to the normal decode path (which errors on the null). formOutputColumn records every + /// row in block_missing_values for this case. + if (physically_nullable && stats.__isset.null_count && stats.null_count == meta_data.num_values + && meta_data.num_values > 0) + { + const bool null_as_default = options.format.null_as_default && !column_info.output_nullable; + if (column_info.output_nullable) + column.constant_value = Null{}; + else if (null_as_default) + column.constant_value = column_info.output_type->getDefault(); + else + return; + + column.is_constant = true; + column.is_all_null = true; + ProfileEvents::increment(ProfileEvents::ParquetConstantColumnChunks); + return; + } + + /// All rows must be non-null. If the parquet column is physically nullable (max definition level + /// > 0), require the statistics to prove zero nulls. If it is REQUIRED (definition level 0) there + /// can be no nulls, so the null_count statistic - which writers commonly omit for non-nullable + /// columns - is unnecessary. A chunk that mixes the value with nulls has two distinct logical + /// values and still needs a null map. + if (physically_nullable && (!stats.__isset.null_count || stats.null_count != 0)) + return; + + if (!stats.__isset.min_value || !stats.__isset.max_value) + return; + if (stats.min_value != stats.max_value) + return; + + /// No truncation guard is needed for the min == max case, including BYTE_ARRAY / + /// FIXED_LEN_BYTE_ARRAY. Statistics bounds are always valid (min_value <= every value <= + /// max_value) and truncation only ever widens them, so a truncated value - or any chunk with two + /// distinct values - yields min_value < max_value. Hence min_value == max_value can only occur for + /// a single value short enough to be stored exactly, so `is_*_value_exact` is implied. + + Field value; + column_info.decoder.decodeField(stats.min_value, /*is_max=*/ false, value); + /// decodeField leaves `value` unchanged (Null) if the physical type is unsupported for stats. + if (value.isNull()) + return; + + column.is_constant = true; + column.constant_value = std::move(value); + ProfileEvents::increment(ProfileEvents::ParquetConstantColumnChunks); +} + +bool Reader::detectConstantSubchunk( + const ColumnChunk & column, const PrimitiveColumnInfo & column_info, + size_t start_row, size_t end_row, size_t row_group_num_rows, + bool & out_all_null, Field & out_value) const +{ + if (column.page_const_info.empty()) + return false; + const size_t num_pages = column.offset_index.page_locations.size(); + if (num_pages == 0 || column.page_const_info.size() != num_pages) + return false; + + bool any = false; + bool all_pages_null = true; + bool all_pages_const_same = true; + Field value; + bool value_set = false; + + for (size_t p = 0; p < num_pages; ++p) + { + const size_t p_start = static_cast(column.offset_index.page_locations[p].first_row_index); + const size_t p_end = (p + 1 < num_pages) + ? static_cast(column.offset_index.page_locations[p + 1].first_row_index) + : row_group_num_rows; + if (p_end <= start_row || p_start >= end_row) + continue; // page does not overlap the subgroup's row range + + any = true; + const auto & pci = column.page_const_info[p]; + if (!pci.all_null) + all_pages_null = false; + if (pci.is_const) + { + if (!value_set) + { + value = pci.value; + value_set = true; + } + else if (value != pci.value) + all_pages_const_same = false; + } + else + { + all_pages_const_same = false; + } + } + + if (!any) + return false; + + if (all_pages_null) + { + /// Every overlapping page is entirely null - mirror detectConstantColumn's all-null branch. + const bool null_as_default = options.format.null_as_default && !column_info.output_nullable; + if (column_info.output_nullable) + out_value = Null{}; + else if (null_as_default) + out_value = column_info.output_type->getDefault(); + else + return false; // non-nullable output without null substitution: let normal decode error + out_all_null = true; + return true; + } + + if (all_pages_const_same && value_set) + { + out_all_null = false; + out_value = value; + return true; + } + + return false; +} + +bool Reader::willFillConstantPages( + const ColumnChunk & column, const PrimitiveColumnInfo & column_info, + const RowGroup & row_group, const RowSubgroup & row_subgroup) const { + if (!options.format.parquet.fill_constant_pages) + return false; + if (column.is_constant) + return false; // whole chunk already constant (tier 1) + if (column.page_const_info.empty()) + return false; // no Column Index info retained + /// PREWHERE / row-level filters and page-pruning predicates are all handled: the fill walks only + /// the rows that pass the filter (row_subgroup.filter), so it produces exactly rows_pass values + /// for any filter, and pruned pages fall outside the subgroup's surviving ranges. The decision + /// here is independent of rows_pass, so it is identical at prefetch time and decode time. + if (!constColumnMaterializationEligible(column_info)) + return false; + + /// We fill the decoded_type column with the output-domain value from the Column Index, which is + /// only valid when no post-decode cast is applied (decoded_type == output value type). + const auto & output_idx = sample_block_to_output_columns_idx.at(column_info.idx_in_output_block); + if (!output_idx.has_value() || output_columns[output_idx.value()].needs_cast) + return false; + + const auto & pages = column.offset_index.page_locations; + const size_t num_pages = pages.size(); + if (num_pages == 0 || column.page_const_info.size() != num_pages) + return false; + + /// Require at least one fillable page (single-value or all-null) overlapping the subgroup. An + /// all-null page is only representable when the output is Nullable or null_as_default substitutes + /// a default; otherwise fall back to the standard decode (which raises the usual not-null error). + const bool null_as_default = options.format.null_as_default && !column_info.output_nullable; + const bool nulls_representable = column_info.output_nullable || null_as_default; + const size_t start = row_subgroup.start_row_idx; + const size_t end = start + row_subgroup.filter.rows_total; + const size_t rg_rows = size_t(row_group.meta->num_rows); + bool any_fillable = false; + for (size_t p = 0; p < num_pages; ++p) + { + const size_t p_start = size_t(pages[p].first_row_index); + const size_t p_end = (p + 1 < num_pages) ? size_t(pages[p + 1].first_row_index) : rg_rows; + if (p_end <= start || p_start >= end) + continue; + const auto & pci = column.page_const_info[p]; + if (pci.all_null && !nulls_representable) + return false; + if (pci.is_const || pci.all_null) + any_fillable = true; + } + return any_fillable; +} + +void Reader::fillConstantPagesAndDecodeRest( + ColumnChunk & column, const PrimitiveColumnInfo & column_info, + ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup) +{ + const size_t start = row_subgroup.start_row_idx; + const size_t rows_total = row_subgroup.filter.rows_total; + const auto & pages = column.offset_index.page_locations; + const size_t num_pages = pages.size(); + const size_t rg_rows = size_t(row_group.meta->num_rows); + const auto & filter = row_subgroup.filter.filter; + + ColumnUInt8::Container * null_map_data = subchunk.null_map + ? &assert_cast(*subchunk.null_map).getData() + : nullptr; + + auto page_end_of = [&](size_t pg) { return (pg + 1 < num_pages) ? size_t(pages[pg + 1].first_row_index) : rg_rows; }; + + /// Iterate the ranges of rows that pass the filter (like the standard row-range decode), and walk + /// the pages overlapping each range. Within a passing range every row is output, so a constant / + /// all-null page contributes exactly its overlapping row count, and a varying page is decoded for + /// that contiguous sub-range. This produces the compact (non-null) values + null map that the + /// caller's expand() / Nullable-wrap / null_as_default tail finalizes - matching the normal + /// decode for any rows_pass, so no rows_pass == rows_total restriction is needed. + size_t page_cursor = 0; // forward cursor over page_locations + size_t row_subidx = 0; // offset within [0, rows_total) + while (true) + { + size_t num_rows = rows_total - row_subidx; + if (!filter.empty()) + { + while (row_subidx < rows_total && !filter[row_subidx]) + ++row_subidx; + num_rows = 0; + while (row_subidx + num_rows < rows_total && filter[row_subidx + num_rows]) + ++num_rows; + } + if (!num_rows) + break; + const size_t range_start = start + row_subidx; + const size_t range_end = range_start + num_rows; + row_subidx += num_rows; + + size_t row = range_start; + while (row < range_end) + { + while (page_cursor < num_pages && page_end_of(page_cursor) <= row) + ++page_cursor; + chassert(page_cursor < num_pages); + const size_t seg_end = std::min(range_end, page_end_of(page_cursor)); + chassert(seg_end > row); + const size_t count = seg_end - row; + const auto & pci = column.page_const_info[page_cursor]; + + if (pci.is_const) + { + /// Fill the single value without reading the page (value is in decoded_type domain, + /// which equals the output value domain - willFillConstantPages required no cast). A + /// single-value page has no nulls, so any null map gets zeros for these rows. + subchunk.column->insertMany(pci.value, count); + if (null_map_data) + null_map_data->resize_fill(null_map_data->size() + count, 0); + } + else if (pci.all_null) + { + /// All-null page: no non-null values; mark these rows null (expand() fills defaults + /// later). null_map exists here (an all-null page implies need_null_map). + chassert(null_map_data); + null_map_data->resize_fill(null_map_data->size() + count, 1); + } + else + { + /// Varying page: decode the contiguous passing sub-range [row, seg_end). All rows in + /// it pass, so no per-row filter is applied here. skipToRowOrNextPage jumps to the page + /// via the offset index, stepping over filled const/all-null pages without loading them. + skipToRowOrNextPage(row, column, column_info); + readRowsInPage(seg_end, subchunk, column, column_info, /*row_subgroup=*/ nullptr); + } + + row = seg_end; + } + } +} + +void Reader::decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info, ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup, MemoryUsageDiff & diff) +{ + /// subchunk.is_constant is set either here from column.is_constant (tier 1, whole chunk) or + /// during subgroup init by detectConstantSubchunk (tier 2, this row subgroup). Either way we skip + /// decoding and let formOutputColumn materialize the value as a ColumnConst. + if (column.is_constant && !subchunk.is_constant) + { + subchunk.is_constant = true; + subchunk.constant_value = column.constant_value; + subchunk.is_all_null = column.is_all_null; + } + + if (subchunk.is_constant) + { + /// This subchunk provably holds a single value in every row - either the whole chunk (tier 1, + /// detectConstantColumn; its data pages were never fetched) or just this subgroup's row range + /// (tier 2, detectConstantSubchunk; the pages are skipped forward by the next subgroup's + /// skipToRowOrNextPage). Skip decoding and hand the value to formOutputColumn, which + /// materializes it directly in the final output type. The value is in the output (post-cast) + /// domain, so it must not go through the decoded_type column and castColumn path. We still run + /// the per-output-column bookkeeping below so the output column is formed once the last of its + /// primitive columns is done. (The subchunk fields were set above for tier 1, or during + /// subgroup init for tier 2.) + OutputColumnState & state = row_subgroup.output.at(column_info.idx_in_output_block); + chassert(!state.column); + size_t prev_count = state.primitive_columns_remaining.fetch_sub(1); + chassert(prev_count > 0); + if (prev_count == 1) + { + const auto & output_idx = sample_block_to_output_columns_idx.at(column_info.idx_in_output_block); + state.column = formOutputColumn(row_subgroup, output_idx.value(), row_subgroup.filter.rows_pass); + } + return; + } + /// Allocate columns for values, null map, and array offsets. size_t output_num_values_estimate = 0; @@ -1363,6 +1841,16 @@ void Reader::decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnIn string_column->getChars().reserve(bytes_to_reserve); } + /// Mixed-topology fill (input_format_parquet_fill_constant_pages): when only part of this + /// subgroup is single-valued, fill the constant pages from the Column Index and decode only the + /// varying ones. Produces a full decoded_type column, so it replaces the decode loop below. + bool filled_constant_pages = false; + if (willFillConstantPages(column, column_info, row_group, row_subgroup)) + { + fillConstantPagesAndDecodeRest(column, column_info, subchunk, row_group, row_subgroup); + filled_constant_pages = true; + } + /// Find ranges of rows that pass filter and decode them. /// When we have per-page prefetches (offset index), some pages may have had their prefetch @@ -1386,7 +1874,11 @@ void Reader::decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnIn !column.need_null_map; const size_t subgroup_end_row_idx = row_subgroup.start_row_idx + row_subgroup.filter.rows_total; - if (use_filter_in_decoder) + if (filled_constant_pages) + { + /// Already produced the whole subchunk column above; skip the normal decode loop. + } + else if (use_filter_in_decoder) { skipToRowOrNextPage(row_subgroup.start_row_idx, column, column_info); @@ -1488,6 +1980,21 @@ void Reader::decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnIn chassert(subchunk.column->getDataType() == column_info.output_type->getColumnType()); + /// The memory charged for this subchunk in scheduleTask was an estimate + /// (estimateColumnMemoryBytesPerRow), which can undershoot for long strings or skewed data, so + /// real RAM would overshoot the watermark and the overshoot compounds with decode-ahead depth. + /// Reconcile the charge up to the actual decoded footprint now, before formOutputColumn (below) + /// may move `subchunk.column` into the output, so the stage counter is honest and the scheduler + /// stops decoding ahead before real RAM exceeds the cap. Grow-only: never drop below the + /// reservation (fail-closed). + size_t actual_bytes = subchunk.column->allocatedBytes(); + for (const auto & offsets : subchunk.arrays_offsets) + if (offsets) + actual_bytes += offsets->allocatedBytes(); + size_t already_charged = subchunk.column_and_offsets_memory.charged(); + if (actual_bytes > already_charged) + subchunk.column_and_offsets_memory.add(actual_bytes - already_charged, &diff); + OutputColumnState & state = row_subgroup.output.at(column_info.idx_in_output_block); chassert(!state.column); size_t prev_count = state.primitive_columns_remaining.fetch_sub(1); @@ -2154,6 +2661,48 @@ MutableColumnPtr Reader::formOutputColumn(RowSubgroup & row_subgroup, size_t out chassert(output_info.primitive_start + 1 == output_info.primitive_end); size_t primitive_idx = output_info.primitive_start; ColumnSubchunk & subchunk = row_subgroup.columns.at(primitive_idx); + + if (subchunk.is_constant) + { + /// Constant column chunk (see detectConstantColumn): the whole chunk is a single value, + /// so materialize it as a ColumnConst rather than an expanded column. This is O(1) instead + /// of O(rows), and the const-ness propagates downstream: a PREWHERE/WHERE predicate + /// computes its result from the value without expanding the stored column, and GROUP BY / + /// aggregation over this column get a const key. + ColumnPtr result; + if (subchunk.is_all_null) + { + /// constant_value was synthesized directly in the output domain (Null for a Nullable + /// output, or the output default under null_as_default), so no cast applies. + MutableColumnPtr single_value = output_info.output_type->createColumn(); + single_value->insert(subchunk.constant_value); + result = ColumnConst::create(std::move(single_value), num_rows); + + /// An all-null chunk must record every row in block_missing_values, matching the + /// normal decode path (which records nulls from the null map); needed for + /// input_format_null_as_default. + if (output_info.idx_in_output_block.has_value() + && *output_info.idx_in_output_block < row_subgroup.block_missing_values.getNumColumns()) + row_subgroup.block_missing_values.setBits(*output_info.idx_in_output_block, num_rows); + } + else + { + /// The value came from decodeField, i.e. the decoded/input domain. Build the const in + /// input_type and apply the same castColumn the per-row decode uses when the output + /// type differs. Today the optimization only fires when the stats decoder does not + /// need a value-transforming conversion (input_type == output_type), so this cast is a + /// no-op; it is here so the constant path stays correct if that ever changes. Casting a + /// ColumnConst is O(1) (one value) and preserves const-ness. + MutableColumnPtr single_value = output_info.input_type->createColumn(); + single_value->insert(subchunk.constant_value); + result = ColumnConst::create(std::move(single_value), num_rows); + if (output_info.needs_cast) + result = castColumn({result, output_info.input_type, output_info.name}, output_info.output_type); + } + + return IColumn::mutate(std::move(result)); + } + res = std::move(subchunk.column); if (output_info.idx_in_output_block.has_value() && diff --git a/src/Processors/Formats/Impl/Parquet/Reader.h b/src/Processors/Formats/Impl/Parquet/Reader.h index 53b598aa5524..09cb32436e77 100644 --- a/src/Processors/Formats/Impl/Parquet/Reader.h +++ b/src/Processors/Formats/Impl/Parquet/Reader.h @@ -297,6 +297,31 @@ struct Reader bool use_column_index = false; bool need_null_map = false; + /// This column chunk provably holds a single repeated value in every row (see + /// detectConstantColumn). When set, we skip prefetching and decoding the data pages and + /// materialize `constant_value` directly instead. `constant_value` is the already-decoded + /// value (not the raw parquet-encoded bytes). + bool is_constant = false; + Field constant_value; + /// Sub-case of `is_constant`: the chunk is provably all-null (null_count == num_values). + /// `constant_value` is then `Null` for a Nullable output, or the output default when + /// `null_as_default` substitutes nulls for a non-nullable output. formOutputColumn also + /// records every row in block_missing_values (the plain constant case has no nulls). + bool is_all_null = false; + + /// Per-page constant info from the Column Index (tier 2; see applyColumnIndex and + /// detectConstantSubchunk). Populated only when the column index is loaded, indexed by data + /// page. `value` is in the output (post-cast) domain. Empty when unavailable. For truncatable + /// physical types (BYTE_ARRAY / FIXED_LEN_BYTE_ARRAY) only `all_null` is trusted, never + /// `is_const` (the Column Index has no per-page exactness flag). + struct PageConstInfo + { + bool is_const = false; /// page holds a single non-null value + bool all_null = false; /// page is entirely null + Field value; /// the value when is_const + }; + std::vector page_const_info; + /// Prefetches. /// TODO [parquet]: Check that all handles and tokens are reset after correct stages. PrefetchHandle bloom_filter_header_prefetch; @@ -347,6 +372,15 @@ struct Reader /// Primitive column. MutableColumnPtr column; + /// Set by decodePrimitiveColumn when the source column chunk is constant (see + /// ColumnChunk::is_constant): `column` is then left empty and formOutputColumn materializes + /// `constant_value` directly in the final output type. `constant_value` is in the output + /// (post-cast) domain. + bool is_constant = false; + Field constant_value; + /// Mirror of ColumnChunk::is_all_null (see there). + bool is_all_null = false; + MutableColumnPtr null_map; /// If this primitive column is inside an array, this is the offsets for `ColumnArray`s at @@ -486,7 +520,9 @@ struct Reader void init(const ReadOptions & options_, const Block & sample_block_, FormatFilterInfoPtr format_filter_info_); - static parq::FileMetaData readFileMetaData(Prefetcher & prefetcher); + /// footer_size_hint: optional hint (bytes) for the initial tail read; 0 = default speculative + /// read (see ReadOptions::footer_metadata_size_hint / estimateParquetFooterSize). + static parq::FileMetaData readFileMetaData(Prefetcher & prefetcher, size_t footer_size_hint = 0); void prefilterAndInitRowGroups(const std::optional> & row_groups_to_read); void preparePrewhere(); @@ -505,12 +541,44 @@ struct Reader /// Call after prewhere is done on row subgroup. Un-requests prefetch for fully filtered out pages, /// adds pages that need prefetch to `out`. Must be called in order. /// May assign dictionary_page_prefetch. - void determinePagesToPrefetch(ColumnChunk & column, const RowSubgroup & row_subgroup, const RowGroup & row_group, std::vector & out); + void determinePagesToPrefetch(ColumnChunk & column, const ColumnSubchunk & subchunk, const PrimitiveColumnInfo & column_info, const RowSubgroup & row_subgroup, const RowGroup & row_group, std::vector & out); /// Guess how much memory ColumnSubchunk::{column, arrays_offsets} will use, per row. double estimateColumnMemoryBytesPerRow(const ColumnChunk & column, const RowGroup & row_group, const PrimitiveColumnInfo & column_info) const; - void decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info, ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup); + void decodePrimitiveColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info, ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup, MemoryUsageDiff & diff); + + /// If the column chunk provably holds one repeated value in every row, sets column.is_constant + /// and column.constant_value. Two sub-cases, both from column chunk statistics (Tier 1): + /// - a single non-null value in every row (min_value == max_value, no nulls), or + /// - an all-null chunk (null_count == num_values), which also sets column.is_all_null. + /// Only applies to flat, top-level primitive columns; see the implementation for the exact + /// conditions. + void detectConstantColumn(ColumnChunk & column, const PrimitiveColumnInfo & column_info) const; + + /// Shared eligibility gate for constant-column materialization (both tiers): the setting is on, + /// stats are decodable, and the leaf is a flat top-level primitive output column. + bool constColumnMaterializationEligible(const PrimitiveColumnInfo & column_info) const; + + /// Tier 2 (per row subgroup): using the retained per-page Column Index info + /// (ColumnChunk::page_const_info), decide whether `column` is constant over the row range + /// [start_row, end_row) covered by a subgroup. Returns true and sets out_all_null / out_value + /// when every overlapping page is all-null, or every overlapping page holds the same single + /// value. row_group_num_rows is the row group's total row count (for the last page's end). + bool detectConstantSubchunk( + const ColumnChunk & column, const PrimitiveColumnInfo & column_info, + size_t start_row, size_t end_row, size_t row_group_num_rows, + bool & out_all_null, Field & out_value) const; + + /// Mixed-topology fill (input_format_parquet_fill_constant_pages): true when this subgroup has + /// some single-value pages (and no all-null pages) that can be filled from the Column Index + /// while the rest is decoded normally. See willFillConstantPages / fillConstantPagesAndDecodeRest. + bool willFillConstantPages( + const ColumnChunk & column, const PrimitiveColumnInfo & column_info, + const RowGroup & row_group, const RowSubgroup & row_subgroup) const; + void fillConstantPagesAndDecodeRest( + ColumnChunk & column, const PrimitiveColumnInfo & column_info, + ColumnSubchunk & subchunk, const RowGroup & row_group, RowSubgroup & row_subgroup); /// Returns mutable column because some of the recursive calls require it, /// e.g. ColumnArray::create does assumeMutable() on the nested columns. diff --git a/src/Processors/Formats/Impl/ParquetV3BlockInputFormat.cpp b/src/Processors/Formats/Impl/ParquetV3BlockInputFormat.cpp index edbf421ccbeb..dcbae5deb127 100644 --- a/src/Processors/Formats/Impl/ParquetV3BlockInputFormat.cpp +++ b/src/Processors/Formats/Impl/ParquetV3BlockInputFormat.cpp @@ -58,6 +58,44 @@ ParquetV3BlockInputFormat::ParquetV3BlockInputFormat( read_options.min_bytes_for_seek = min_bytes_for_seek; read_options.bytes_per_read_task = min_bytes_for_seek * 4; + /// A data lake (e.g. Iceberg) may have precomputed how much of the file tail to read for the + /// footer from its per-file stats; use it to size the initial metadata read (see + /// estimateParquetFooterSize). 0/absent keeps the default speculative read. + if (object_with_metadata && object_with_metadata->footer_size_hint) + read_options.footer_metadata_size_hint = *object_with_metadata->footer_size_hint; + + /// Multipart part boundaries (from GetObjectAttributes via the identity cache), used to align + /// coalesced reads to part boundaries. EXPERIMENTAL: for measuring part-aligned reads. Gated by + /// input_format_parquet_align_reads_to_multipart_boundaries so it can be A/B-compared; when off, + /// the offsets stay empty and the Prefetcher coalesces as usual. + if (read_options.format.parquet.align_reads_to_multipart_boundaries + && object_with_metadata && object_with_metadata->metadata && !object_with_metadata->metadata->part_offsets.empty()) + read_options.multipart_part_offsets.assign( + object_with_metadata->metadata->part_offsets.begin(), + object_with_metadata->metadata->part_offsets.end()); + + /// Fixed-grid alignment stride (used when real per-file part offsets are unavailable). Default to + /// 16 MiB when unset: it's the S3 multipart part size of ClickHouse's own writer (whose adaptive + /// 16/32/64 MiB ramp keeps every part boundary a multiple of 16 MiB) and of Hadoop S3A (64/128), + /// so a 16 MiB grid never straddles a real boundary for those (exact for 16 MiB parts, harmless + /// over-split for bigger); ~50% fewer straddles for 8 MiB (aws cli). Only misses non-16 sizes + /// (e.g. delta-rs 10 MiB) - set input_format_parquet_read_alignment_bytes explicitly for those. + read_options.read_alignment_stride = read_options.format.parquet.read_alignment_bytes + ? read_options.format.parquet.read_alignment_bytes + : (16ul << 20); + read_options.read_alignment_min_bytes = read_options.format.parquet.read_alignment_min_bytes; + + /// Split boundary-straddling reads into parallel per-part segments (+ speculative-parallel footer). + read_options.split_reads_across_boundaries = read_options.format.parquet.split_reads_across_part_boundaries; + + /// Anti-amplification: cap how much filler coalescing may read to bridge gaps in a sparse column. + read_options.read_min_fill_ratio = read_options.format.parquet.read_min_fill_ratio; + + /// Hedged reads (tail-latency mitigation). + read_options.hedged_read_threshold_ms = read_options.format.parquet.hedged_read_threshold_ms; + read_options.hedged_read_max_bytes = read_options.format.parquet.hedged_read_max_bytes; + read_options.hedged_read_max_inflight = read_options.format.parquet.hedged_read_max_inflight; + if (!format_filter_info) format_filter_info = std::make_shared(); } @@ -112,11 +150,11 @@ parquet::format::FileMetaData ParquetV3BlockInputFormat::getFileMetadata(Parquet String etag = object_with_metadata->metadata->etag; ParquetMetadataCacheKey cache_key = ParquetMetadataCache::createKey(file_name, etag); return metadata_cache->getOrSetMetadata( - cache_key, [&]() { return Parquet::Reader::readFileMetaData(prefetcher); }); + cache_key, [&]() { return Parquet::Reader::readFileMetaData(prefetcher, read_options.footer_metadata_size_hint); }); } else { - return Parquet::Reader::readFileMetaData(prefetcher); + return Parquet::Reader::readFileMetaData(prefetcher, read_options.footer_metadata_size_hint); } } @@ -214,7 +252,7 @@ void NativeParquetSchemaReader::initializeIfNeeded() return; Parquet::Prefetcher prefetcher; prefetcher.init(&in, read_options, /*parser_shared_resources_=*/ nullptr); - file_metadata = Parquet::Reader::readFileMetaData(prefetcher); + file_metadata = Parquet::Reader::readFileMetaData(prefetcher, read_options.footer_metadata_size_hint); initialized = true; } diff --git a/src/Storages/ObjectStorage/DataLakes/Common/AvroForIcebergDeserializer.cpp b/src/Storages/ObjectStorage/DataLakes/Common/AvroForIcebergDeserializer.cpp index 4517834d0dec..c41593345828 100644 --- a/src/Storages/ObjectStorage/DataLakes/Common/AvroForIcebergDeserializer.cpp +++ b/src/Storages/ObjectStorage/DataLakes/Common/AvroForIcebergDeserializer.cpp @@ -242,6 +242,16 @@ ParsedManifestFileEntryPtr AvroForIcebergDeserializer::createParsedManifestFileE const auto record_count = getValueFromRowByName(row_index, c_data_file_record_count, TypeIndex::Int64).safeGet(); const auto file_size_in_bytes = getValueFromRowByName(row_index, c_data_file_file_size_in_bytes, TypeIndex::Int64).safeGet(); + /// Optional Parquet row-group split offsets - used only to size the footer tail read. + std::vector split_offsets; + if (hasPath(c_data_file_split_offsets)) + { + Field so = getValueFromRowByName(row_index, c_data_file_split_offsets); + if (!so.isNull()) + for (const auto & off : so.safeGet()) + split_offsets.push_back(off.safeGet()); + } + switch (content_type) { case FileContentType::DATA: { @@ -261,7 +271,8 @@ ParsedManifestFileEntryPtr AvroForIcebergDeserializer::createParsedManifestFileE /*equality_ids*/ std::nullopt, sort_order_id, record_count, - file_size_in_bytes); + file_size_in_bytes, + std::move(split_offsets)); } case FileContentType::POSITION_DELETE: { /// reference_file_path can be absent in schema for some reason, though it is present in specification: https://iceberg.apache.org/spec/#manifests diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h b/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h index b87ff718e172..640d573a31fc 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h @@ -63,6 +63,7 @@ DEFINE_ICEBERG_FIELD(added_rows_count); DEFINE_ICEBERG_FIELD(existing_rows_count); DEFINE_ICEBERG_FIELD(deleted_rows_count); DEFINE_ICEBERG_FIELD(record_count); +DEFINE_ICEBERG_FIELD(split_offsets); DEFINE_ICEBERG_FIELD(file_path); DEFINE_ICEBERG_FIELD(file_format); DEFINE_ICEBERG_FIELD(file_size_in_bytes); @@ -178,6 +179,7 @@ DEFINE_ICEBERG_FIELD_COMPOUND(data_file, upper_bounds); DEFINE_ICEBERG_FIELD_COMPOUND(data_file, referenced_data_file); DEFINE_ICEBERG_FIELD_COMPOUND(data_file, sort_order_id); DEFINE_ICEBERG_FIELD_COMPOUND(data_file, record_count); +DEFINE_ICEBERG_FIELD_COMPOUND(data_file, split_offsets); DEFINE_ICEBERG_FIELD_COMPOUND(data_file, file_size_in_bytes); /// Fallback defaults for snapshot retention policy when table properties are absent. diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDataObjectInfo.cpp b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDataObjectInfo.cpp index 4e161ca8c863..8299972bf338 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDataObjectInfo.cpp +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDataObjectInfo.cpp @@ -16,6 +16,11 @@ #include #include +#if USE_PARQUET +#include +#include +#endif + namespace DB::ErrorCodes { extern const int NOT_IMPLEMENTED; @@ -52,6 +57,16 @@ String computePartitionId(const Row & partition_key_value) #if USE_AVRO +#if USE_PARQUET +/// Rough byte size of a decoded manifest bound value, used only to size the parquet footer read. +static size_t estimateBoundFieldBytes(const Field & f) +{ + if (f.getType() == Field::Types::String) + return f.safeGet().size(); + return 16; /// conservative for numeric / decimal / date / uuid bounds +} +#endif + IcebergDataObjectInfo::IcebergDataObjectInfo( Iceberg::ProcessedManifestFileEntryPtr data_manifest_file_entry_, const String & resolved_storage_path_, Int32 schema_id_relevant_to_iterator_) : ObjectInfo(RelativePathWithMetadata(resolved_storage_path_)) @@ -68,6 +83,37 @@ IcebergDataObjectInfo::IcebergDataObjectInfo( data_manifest_file_entry_->parsed_entry->record_count, data_manifest_file_entry_->parsed_entry->file_size_in_bytes} { + /// Note: object identity (size / etag / multipart part offsets) is intentionally NOT pre-populated + /// from the manifest here. It is fetched once via GetObjectAttributes and cached by the + /// object-storage identity cache (see S3ObjectStorage::getObjectMetadata / ObjectStorageIdentityCache), + /// so all object reads - Iceberg data files included - go through one path that yields the real + /// ETag (needed by the filesystem / page / parquet-metadata caches) and the multipart layout + /// (needed for read alignment). Pre-populating metadata here would skip that path and lose both. +#if USE_PARQUET + /// Precompute a footer-size hint from the manifest stats so the parquet reader can fetch the + /// FileMetaData in a single tail read (see Parquet::estimateParquetFooterSize). Parquet only; + /// the hint is ignored by other formats. The row-group count comes from the manifest's + /// split_offsets (one per row group) when present - accurate even for byte-split row groups - + /// and falls back to a record_count/1M guess otherwise. Only affects read count, never correctness. + const auto & entry = *data_manifest_file_entry_->parsed_entry; + if (entry.file_format == "PARQUET") + { + size_t num_columns = std::max(entry.columns_infos.size(), entry.value_bounds.size()); + if (num_columns > 0) + { + constexpr size_t rows_per_row_group_guess = 1'000'000; + size_t rows = size_t(std::max(entry.record_count, 0)); + size_t num_row_groups = !entry.split_offsets.empty() + ? entry.split_offsets.size() + : std::max(1, (rows + rows_per_row_group_guess - 1) / rows_per_row_group_guess); + size_t bounds_bytes = 0; + for (const auto & [field_id, bounds] : entry.value_bounds) + bounds_bytes += estimateBoundFieldBytes(bounds.first) + estimateBoundFieldBytes(bounds.second); + relative_path_with_metadata.footer_size_hint + = Parquet::estimateParquetFooterSize(num_columns, num_row_groups, bounds_bytes); + } + } +#endif } IcebergDataObjectInfo::IcebergDataObjectInfo(const RelativePathWithMetadata & path_) diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp index a7bd62ccd110..293328cea8d7 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp @@ -169,11 +169,24 @@ Iceberg::PersistentTableComponents IcebergMetadata::initializePersistentTableCom ContextPtr context_, LoggerPtr log) { + /// A catalog (e.g. Iceberg REST) already knows the table's Iceberg table-uuid from its LoadTable + /// response and forwards it as the iceberg_metadata_table_uuid storage setting (see + /// DatabaseDataLake / RestCatalog::setTableUUID). When present, use it to key the metadata cache + /// on this very first read, so the bootstrap read stores the metadata under (table_uuid, path) + /// and the later per-query read hits instead of re-reading the same metadata file. When absent + /// (bare-path table function, or Hadoop / version-hint tables with no catalog) the uuid can only + /// come from the file itself, so this stays nullopt and getMetadataJSONObject seeds the cache + /// after parsing. + const auto & data_lake_settings = configuration->getDataLakeSettings(); + std::optional known_table_uuid; + if (data_lake_settings[DataLakeStorageSetting::iceberg_metadata_table_uuid].changed) + known_table_uuid = normalizeUuid(data_lake_settings[DataLakeStorageSetting::iceberg_metadata_table_uuid].value); + const auto [metadata_version, metadata_file_path, compression_method] - = getLatestOrExplicitMetadataFileAndVersion(object_storage, configuration->getPathForRead().path, configuration->getDataLakeSettings(), cache_ptr, context_, log.get(), std::nullopt, CompressionMethod::None, true); + = getLatestOrExplicitMetadataFileAndVersion(object_storage, configuration->getPathForRead().path, data_lake_settings, cache_ptr, context_, log.get(), known_table_uuid, CompressionMethod::None, true); LOG_DEBUG(log, "Latest metadata file path is {}, version {}", metadata_file_path, metadata_version); auto metadata_object - = getMetadataJSONObject(metadata_file_path, object_storage, cache_ptr, context_, log, compression_method, std::nullopt); + = getMetadataJSONObject(metadata_file_path, object_storage, cache_ptr, context_, log, compression_method, known_table_uuid); Int32 format_version = metadata_object->getValue(f_format_version); String table_location = metadata_object->getValue(f_location); std::optional table_uuid = std::nullopt; diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp index 5bc1fe22fbce..7a267e3b2b6d 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/ManifestFile.h b/src/Storages/ObjectStorage/DataLakes/Iceberg/ManifestFile.h index f7c2ced00bf3..7f32178bc24f 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/ManifestFile.h +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/ManifestFile.h @@ -94,6 +94,10 @@ struct ParsedManifestFileEntry : boost::noncopyable Int64 record_count; Int64 file_size_in_bytes; + /// Optional per-file row-group split offsets (Parquet: byte offset of each row group). Used only + /// to size the parquet footer tail read (see IcebergDataObjectInfo). Empty if not present. + std::vector split_offsets; + ParsedManifestFileEntry( FileContentType content_type_, IcebergPathFromMetadata file_path_key_, @@ -110,7 +114,8 @@ struct ParsedManifestFileEntry : boost::noncopyable std::optional> equality_ids_, std::optional sort_order_id_, Int64 record_count_, - Int64 file_size_in_bytes_) + Int64 file_size_in_bytes_, + std::vector split_offsets_ = {}) : content_type(content_type_) , file_path_key(std::move(file_path_key_)) , row_number(row_number_) @@ -127,6 +132,7 @@ struct ParsedManifestFileEntry : boost::noncopyable , sort_order_id(sort_order_id_) , record_count(record_count_) , file_size_in_bytes(file_size_in_bytes_) + , split_offsets(std::move(split_offsets_)) { } }; diff --git a/src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp b/src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp index 3ac3042dc50c..6c0abbe885e9 100644 --- a/src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp +++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp @@ -502,7 +502,24 @@ Poco::JSON::Object::Ptr getMetadataJSONObject( Poco::JSON::Parser parser; /// For some reason base/base/JSON.h can not parse this json file Poco::Dynamic::Var json = parser.parse(metadata_json_str); - return json.extract(); + auto metadata_object = json.extract(); + + /// The very first read of a table's metadata (bootstrap, in initializePersistentTableComponents) + /// is issued with no table_uuid - the uuid is only learned by parsing this very file - so it + /// takes the uncached branch above and stores nothing. Every later read keys the cache on + /// (table_uuid, path) and would therefore miss and re-read the same file. Seed the cache here, + /// using the uuid we just parsed, so those later reads hit. Only the top-level table metadata + /// carries `table-uuid`; other JSON (e.g. none) is skipped. Immutable per (uuid, path), so this + /// is safe and preserves the table-recreate guard (a recreated table has a different uuid). + if (metadata_cache && !table_uuid.has_value() && metadata_object->has(f_table_uuid)) + { + auto parsed_uuid = normalizeUuid(metadata_object->getValue(f_table_uuid)); + metadata_cache->getOrSetTableMetadata( + IcebergMetadataFilesCache::getKey(parsed_uuid, metadata_file_path), + [&metadata_json_str]() { return metadata_json_str; }); + } + + return metadata_object; } /// Returns type and required diff --git a/src/Storages/ObjectStorage/StorageObjectStorageSource.cpp b/src/Storages/ObjectStorage/StorageObjectStorageSource.cpp index d004243b3131..cc98ab9600f4 100644 --- a/src/Storages/ObjectStorage/StorageObjectStorageSource.cpp +++ b/src/Storages/ObjectStorage/StorageObjectStorageSource.cpp @@ -788,7 +788,10 @@ StorageObjectStorageSource::ReaderHolder StorageObjectStorageSource::createReade { ProfileEvents::increment(ProfileEvents::ObjectStorageReadObjects); compression_method = chooseCompressionMethod(object_info->getFileName(), configuration->compression_method); - read_buf = createReadBuffer(object_info->relative_path_with_metadata, object_storage, context_, log); + const bool format_is_random_access = FormatFactory::instance().checkIfFormatIsRandomAccessInput(format_name); + read_buf = createReadBuffer( + object_info->relative_path_with_metadata, object_storage, context_, log, + /*read_settings=*/ std::nullopt, format_is_random_access); } Block initial_header = read_from_format_info.format_header; @@ -1099,7 +1102,8 @@ std::unique_ptr createReadBuffer( const ObjectStoragePtr & object_storage, const ContextPtr & context_, const LoggerPtr & log, - const std::optional & read_settings) + const std::optional & read_settings, + bool format_is_random_access) { const auto & settings = context_->getSettingsRef(); const auto & effective_read_settings = read_settings.has_value() ? read_settings.value() : context_->getReadSettings(); @@ -1166,8 +1170,20 @@ std::unique_ptr createReadBuffer( // Create a read buffer that will prefetch the first ~1 MB of the file. // When reading lots of tiny files, this prefetching almost doubles the throughput. // For bigger files, parallel reading is more useful. - const bool object_too_small = is_size_known - && object_size <= 2 * context_->getSettingsRef()[Setting::max_download_buffer_size]; + // + // The prefetch reads from the start of the file. Row/streaming formats (CSV, JSON, ...) consume + // from the start, so it is always useful for them. Column-oriented random-access formats + // (Parquet/ORC/Arrow) instead read the footer at the *tail* first and drive their own + // prefetcher: a from-start read-ahead that does not span the whole file is then dropped and + // wasted (an extra object read that transfers bytes nobody consumes). So for those formats only + // prefetch when the whole object fits a single download buffer - then it degenerates to one + // whole-file read the format serves entirely from memory (footer included). Above that, leave + // reading to the format's own tail-first prefetcher. + const size_t max_download_buffer_size = context_->getSettingsRef()[Setting::max_download_buffer_size]; + const size_t prefetch_size_limit = format_is_random_access + ? max_download_buffer_size + : 2 * max_download_buffer_size; + const bool object_too_small = is_size_known && object_size <= prefetch_size_limit; const bool use_prefetch = object_too_small && modified_read_settings.remote_fs_settings.method == RemoteFSReadMethod::threadpool && modified_read_settings.remote_fs_settings.prefetch; diff --git a/src/Storages/ObjectStorage/Utils.h b/src/Storages/ObjectStorage/Utils.h index 931ebcbed9ac..8c4c6270389c 100644 --- a/src/Storages/ObjectStorage/Utils.h +++ b/src/Storages/ObjectStorage/Utils.h @@ -34,7 +34,11 @@ std::unique_ptr createReadBuffer( const ObjectStoragePtr & object_storage, const ContextPtr & context_, const LoggerPtr & log, - const std::optional & read_settings = std::nullopt); + const std::optional & read_settings = std::nullopt, + /// Set when the consuming input format is column-oriented / random-access (Parquet/ORC/Arrow): + /// suppresses the generic from-start read-ahead unless the whole object fits one buffer, so the + /// format's own tail-first prefetcher is not shadowed by a wasted read. See createReadBuffer. + bool format_is_random_access = false); ASTs::iterator getFirstKeyValueArgument(ASTs & args); std::unordered_map parseKeyValueArguments(const ASTs & function_args, ContextPtr context); diff --git a/tests/integration/test_database_delta/test.py b/tests/integration/test_database_delta/test.py index 76ec56858eec..d4c274972df6 100644 --- a/tests/integration/test_database_delta/test.py +++ b/tests/integration/test_database_delta/test.py @@ -13,6 +13,9 @@ UC_LOG = "/var/lib/clickhouse/user_files/unitycatalog/uc.log" +CATALOG_NAME = "unity_catalog_test_db" + + def start_unity_catalog(node): node.exec_in_container( [ @@ -1026,3 +1029,42 @@ def test_varchar_char_types_via_unity_catalog(started_cluster, use_delta_kernel) .strip() ) assert row == "1\thello varchar\thello char" + + +def test_namespace_filter(started_cluster): + node = started_cluster.instances["node1"] + + # Use the same table name in all namespaces + table_name = f"table_{uuid.uuid4()}".replace("-", "_") + namespace_prefix = f"namespace_{uuid.uuid4()}_".replace("-", "_") + + + def create_namespace(suffix): + namespace = f"{namespace_prefix}{suffix}" + execute_spark_query( + node, f"CREATE SCHEMA {namespace}" + ) + execute_spark_query( + node, f"CREATE TABLE {namespace}.{table_name} (col1 int, col2 double) using Delta location '/var/lib/clickhouse/user_files/tmp/{namespace}/{table_name}'" + ) + + create_namespace("alpha"); + create_namespace("bravo"); + + node.query( + f""" + drop database if exists {CATALOG_NAME}; + create database {CATALOG_NAME} + engine DataLakeCatalog('http://localhost:8080/api/2.1/unity-catalog') + settings warehouse = 'unity', catalog_type='unity', vended_credentials=false, namespaces = '{namespace_prefix}alpha' + """, + settings={"allow_database_unity_catalog": "1"}, + ) + + assert node.query(f"SELECT name FROM system.tables WHERE database='{CATALOG_NAME}' ORDER BY name", settings={"show_data_lake_catalogs_in_system_tables": 1}) == TSV( + [ + [f"{namespace_prefix}alpha.{table_name}"], + ]) + + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}alpha.{table_name}`") == "0\n" + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}bravo.{table_name}`") diff --git a/tests/integration/test_database_glue/test.py b/tests/integration/test_database_glue/test.py index 96a5eddcb7a2..1018b0d3193d 100644 --- a/tests/integration/test_database_glue/test.py +++ b/tests/integration/test_database_glue/test.py @@ -16,6 +16,7 @@ from pyiceberg.table.sorting import SortField, SortOrder from pyiceberg.transforms import DayTransform, IdentityTransform from helpers.config_cluster import minio_access_key, minio_secret_key +from helpers.test_tools import TSV import decimal from pyiceberg.types import ( DoubleType, @@ -1129,6 +1130,45 @@ def test_check_database(started_cluster): "SYSTEM DISABLE FAILPOINT check_database_datalake_negative" ) + +def test_namespace_filter(started_cluster): + node = started_cluster.instances["node1"] + + # Use the same table name in all namespaces + table_name = f"table_{uuid.uuid4()}" + table2_name = f"table2_{uuid.uuid4()}" + namespace_prefix = f"namespace_{uuid.uuid4()}_" + + catalog = load_catalog_impl(started_cluster) + + def create_namespace(suffix): + namespace = f"{namespace_prefix}{suffix}" + catalog.create_namespace(namespace) + create_table(catalog, namespace, table_name, DEFAULT_SCHEMA, PartitionSpec(), DEFAULT_SORT_ORDER) + + create_namespace("alpha"); + create_namespace("bravo"); + + create_clickhouse_glue_database(started_cluster, node, CATALOG_NAME, + additional_settings={ + "namespaces": f"{namespace_prefix}alpha" + }) + + assert node.query(f"SELECT name FROM system.tables WHERE database='{CATALOG_NAME}' ORDER BY name", settings={"show_data_lake_catalogs_in_system_tables": 1}) == TSV( + [ + [f"{namespace_prefix}alpha.{table_name}"], + ]) + + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}alpha.{table_name}`") == "0\n" + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}bravo.{table_name}`") + + node.query(f"CREATE TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.{table2_name}` (x String) ENGINE = IcebergS3('http://minio:9000/warehouse-glue/{namespace_prefix}alpha/a1/{table2_name}/', '{minio_access_key}', '{minio_secret_key}')") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"CREATE TABLE {CATALOG_NAME}.`{namespace_prefix}bravo.{table2_name}` (x String) ENGINE = IcebergS3('http://minio:9000/warehouse-glue/{namespace_prefix}bravo/{table2_name}/', '{minio_access_key}', '{minio_secret_key}')") + + node.query(f"DROP TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.{table_name}`") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"DROP TABLE {CATALOG_NAME}.`{namespace_prefix}bravo.{table_name}`") + + def test_sts_smoke(started_cluster): """Test that STS authentication works with Glue catalog using role_arn and role_session_name""" node = started_cluster.instances["node1"] diff --git a/tests/integration/test_database_iceberg/test.py b/tests/integration/test_database_iceberg/test.py index 5d233f89d7ca..d48f68a36997 100644 --- a/tests/integration/test_database_iceberg/test.py +++ b/tests/integration/test_database_iceberg/test.py @@ -1124,6 +1124,7 @@ def add_column(idx): assert node.query(f"SELECT {select_cols} FROM {table_ref} ORDER BY ALL", settings=write_settings) == expected + def test_gcs(started_cluster): node = started_cluster.instances["node1"] @@ -1144,6 +1145,72 @@ def test_gcs(started_cluster): assert "Google cloud storage converts to S3" in str(err.value) +def test_namespace_filter(started_cluster): + node = started_cluster.instances["node1"] + + # Use the same table name in all namespaces + table_name = f"table_{uuid.uuid4()}" + table2_name = f"table2_{uuid.uuid4()}" + namespace_prefix = f"namespace_{uuid.uuid4()}_" + + catalog = load_catalog_impl(started_cluster) + + def create_namespace(suffix): + namespace = f"{namespace_prefix}{suffix}" + catalog.create_namespace(namespace) + create_table(catalog, namespace, table_name, DEFAULT_SCHEMA, PartitionSpec(), DEFAULT_SORT_ORDER) + + create_namespace("alpha"); + create_namespace("alpha.a1"); + create_namespace("alpha.a2"); + create_namespace("bravo"); + create_namespace("bravo.b1"); + create_namespace("charlie"); + create_namespace("charlie.c1"); + create_namespace("delta"); + create_namespace("delta.d1"); + create_namespace("delta.d2"); + create_namespace("echo"); + create_namespace("echo.e1"); + + create_clickhouse_iceberg_database(started_cluster, node, CATALOG_NAME, + additional_settings={ + "namespaces": f"{namespace_prefix}alpha,{namespace_prefix}alpha.a1,{namespace_prefix}bravo,{namespace_prefix}bravo.*,{namespace_prefix}charlie,{namespace_prefix}delta.d1,{namespace_prefix}echo.*" + }) + + assert node.query(f"SELECT name FROM system.tables WHERE database='{CATALOG_NAME}' ORDER BY name", settings={"show_data_lake_catalogs_in_system_tables": 1}) == TSV( + [ + [f"{namespace_prefix}alpha.a1.{table_name}"], + [f"{namespace_prefix}alpha.{table_name}"], + [f"{namespace_prefix}bravo.b1.{table_name}"], + [f"{namespace_prefix}bravo.{table_name}"], + [f"{namespace_prefix}charlie.{table_name}"], + [f"{namespace_prefix}delta.d1.{table_name}"], + [f"{namespace_prefix}echo.e1.{table_name}"], + ]) + + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}alpha.{table_name}`") == "0\n" + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}alpha.a1.{table_name}`") == "0\n" + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}alpha.a2.{table_name}`") + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}bravo.{table_name}`") == "0\n" + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}bravo.b1.{table_name}`") == "0\n" + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}charlie.{table_name}`") == "0\n" + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}charlie.c1.{table_name}`") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}delta.{table_name}`") + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}delta.d1.{table_name}`") == "0\n" + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}delta.d2.{table_name}`") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}echo.{table_name}`") + assert node.query(f"SELECT count() FROM {CATALOG_NAME}.`{namespace_prefix}echo.e1.{table_name}`") == "0\n" + + node.query(f"CREATE TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.{table2_name}` (x String) ENGINE = IcebergS3('http://minio:9000/warehouse-rest/{namespace_prefix}alpha/{table2_name}/', '{minio_access_key}', '{minio_secret_key}')") + node.query(f"CREATE TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.a1.{table2_name}` (x String) ENGINE = IcebergS3('http://minio:9000/warehouse-rest/{namespace_prefix}alpha/a1/{table2_name}/', '{minio_access_key}', '{minio_secret_key}')") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"CREATE TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.a2.{table2_name}` (x String) ENGINE = IcebergS3('http://minio:9000/warehouse-rest/{namespace_prefix}alpha/a2/{table2_name}/', '{minio_access_key}', '{minio_secret_key}')") + + node.query(f"DROP TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.{table_name}`") + node.query(f"DROP TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.a1.{table_name}`") + assert "is filtered by `namespaces` database parameter." in node.query_and_get_error(f"DROP TABLE {CATALOG_NAME}.`{namespace_prefix}alpha.a2.{table_name}`") + + def test_invalid_auth_header_format(started_cluster): node = started_cluster.instances["node1"] diff --git a/tests/queries/0_stateless/01271_show_privileges.reference b/tests/queries/0_stateless/01271_show_privileges.reference index cc318eca54d0..973da147bafd 100644 --- a/tests/queries/0_stateless/01271_show_privileges.reference +++ b/tests/queries/0_stateless/01271_show_privileges.reference @@ -128,6 +128,7 @@ SYSTEM DROP MARK CACHE ['SYSTEM CLEAR MARK CACHE','SYSTEM DROP MARK','DROP MARK SYSTEM DROP ICEBERG METADATA CACHE ['SYSTEM CLEAR ICEBERG_METADATA_CACHE','SYSTEM DROP ICEBERG_METADATA_CACHE'] GLOBAL SYSTEM DROP CACHE SYSTEM DROP AVRO SCHEMA CACHE ['SYSTEM CLEAR AVRO SCHEMA CACHE','SYSTEM DROP AVRO SCHEMA CACHE','DROP AVRO SCHEMA CACHE'] GLOBAL SYSTEM DROP CACHE SYSTEM DROP PARQUET METADATA CACHE ['SYSTEM DROP PARQUET_METADATA_CACHE'] GLOBAL SYSTEM DROP CACHE +SYSTEM DROP OBJECT STORAGE IDENTITY CACHE ['SYSTEM CLEAR OBJECT STORAGE IDENTITY CACHE','SYSTEM DROP OBJECT STORAGE IDENTITY CACHE'] GLOBAL SYSTEM DROP CACHE SYSTEM PREWARM PRIMARY INDEX CACHE ['SYSTEM PREWARM PRIMARY INDEX','PREWARM PRIMARY INDEX CACHE','PREWARM PRIMARY INDEX'] GLOBAL SYSTEM DROP CACHE SYSTEM DROP PRIMARY INDEX CACHE ['SYSTEM CLEAR PRIMARY INDEX CACHE','SYSTEM DROP PRIMARY INDEX','DROP PRIMARY INDEX CACHE','DROP PRIMARY INDEX'] GLOBAL SYSTEM DROP CACHE SYSTEM DROP UNCOMPRESSED CACHE ['SYSTEM CLEAR UNCOMPRESSED CACHE','SYSTEM DROP UNCOMPRESSED','DROP UNCOMPRESSED CACHE','DROP UNCOMPRESSED'] GLOBAL SYSTEM DROP CACHE diff --git a/tests/queries/0_stateless/04811_parquet_constant_column_optimization.reference b/tests/queries/0_stateless/04811_parquet_constant_column_optimization.reference new file mode 100644 index 000000000000..3328031ec73e --- /dev/null +++ b/tests/queries/0_stateless/04811_parquet_constant_column_optimization.reference @@ -0,0 +1,12 @@ +-- values, optimization on +42 hello 2020-01-02 03:04:05 7 1000 +-- values, optimization off (must be identical) +42 hello 2020-01-02 03:04:05 7 1000 +-- the varying column is read correctly (not treated as constant) +499500 0 999 1000 +-- filters on a constant column still work +1000 +0 +-- optimization fired only when enabled +1 +1 diff --git a/tests/queries/0_stateless/04811_parquet_constant_column_optimization.sh b/tests/queries/0_stateless/04811_parquet_constant_column_optimization.sh new file mode 100755 index 000000000000..2a059de02187 --- /dev/null +++ b/tests/queries/0_stateless/04811_parquet_constant_column_optimization.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# Tags: no-fasttest + +CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +# shellcheck source=../shell_config.sh +. "$CUR_DIR"/../shell_config.sh + +USER_FILES_PATH=$($CLICKHOUSE_CLIENT_BINARY --query "select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')" 2>&1 | grep Exception | awk '{gsub("/nonexist.txt","",$9); print $9}') +WORKING_DIR="${USER_FILES_PATH}/${CLICKHOUSE_TEST_UNIQUE_NAME}" +mkdir -p "${WORKING_DIR}" +DATA_FILE="${WORKING_DIR}/const.parquet" + +# 1000 rows, 100 rows per row group => 10 row groups. `k` varies; the other four columns each hold a +# single value in every row, so their per-chunk min/max statistics have min == max and no nulls. +# `c_dt` is written as TIMESTAMP_MILLIS and read back with a DateTime hint, exercising the +# milliseconds -> seconds stats conversion (the value is in the post-cast output domain). +${CLICKHOUSE_CLIENT} -q " + INSERT INTO FUNCTION file('${DATA_FILE}', Parquet) + SELECT + number AS k, + 42::Int64 AS c_int, + 'hello' AS c_str, + toDateTime('2020-01-02 03:04:05') AS c_dt, + 7::Nullable(Int64) AS c_nullable + FROM numbers(1000) + SETTINGS engine_file_truncate_on_insert = 1, output_format_parquet_row_group_size = 100 +" + +STRUCTURE="k UInt64, c_int Int64, c_str String, c_dt DateTime, c_nullable Nullable(Int64)" + +qid_on="${CLICKHOUSE_TEST_UNIQUE_NAME}_on" +qid_off="${CLICKHOUSE_TEST_UNIQUE_NAME}_off" + +echo "-- values, optimization on" +${CLICKHOUSE_CLIENT} --query_id="${qid_on}" -q " + SELECT c_int, c_str, c_dt, c_nullable, count() + FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') + GROUP BY 1, 2, 3, 4 +" + +echo "-- values, optimization off (must be identical)" +${CLICKHOUSE_CLIENT} --query_id="${qid_off}" -q " + SELECT c_int, c_str, c_dt, c_nullable, count() + FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') + GROUP BY 1, 2, 3, 4 + SETTINGS input_format_parquet_use_constant_column_optimization = 0 +" + +echo "-- the varying column is read correctly (not treated as constant)" +${CLICKHOUSE_CLIENT} -q "SELECT sum(k), min(k), max(k), count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}')" + +echo "-- filters on a constant column still work" +${CLICKHOUSE_CLIENT} -q "SELECT count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') WHERE c_int = 42" +${CLICKHOUSE_CLIENT} -q "SELECT count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') WHERE c_int = 43" + +echo "-- optimization fired only when enabled" +${CLICKHOUSE_CLIENT} -q " + SYSTEM FLUSH LOGS query_log; + SELECT ProfileEvents['ParquetConstantColumnChunks'] > 0 + FROM system.query_log + WHERE event_date >= yesterday() AND event_time >= now() - 600 + AND query_id = '${qid_on}' AND type = 'QueryFinish' AND current_database = currentDatabase(); + SELECT ProfileEvents['ParquetConstantColumnChunks'] = 0 + FROM system.query_log + WHERE event_date >= yesterday() AND event_time >= now() - 600 + AND query_id = '${qid_off}' AND type = 'QueryFinish' AND current_database = currentDatabase(); +" + +rm -rf "${WORKING_DIR}" diff --git a/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.reference b/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.reference new file mode 100644 index 000000000000..dfda8ff8759c --- /dev/null +++ b/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.reference @@ -0,0 +1,14 @@ +-- all-null column, optimization on +1 1000 +-- all-null column, optimization off (must be identical) +1 1000 +-- the varying column is read correctly (not treated as constant) +499500 0 999 1000 +-- IS NULL / IS NOT NULL filters on an all-null column +1000 +0 +-- null_as_default: a non-nullable hint substitutes the default (0) for every row +0 1000 +-- optimization fired only when enabled +1 +1 diff --git a/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.sh b/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.sh new file mode 100755 index 000000000000..21522716651d --- /dev/null +++ b/tests/queries/0_stateless/04812_parquet_all_null_column_optimization.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# Tags: no-fasttest + +CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +# shellcheck source=../shell_config.sh +. "$CUR_DIR"/../shell_config.sh + +USER_FILES_PATH=$($CLICKHOUSE_CLIENT_BINARY --query "select _path,_file from file('nonexist.txt', 'CSV', 'val1 char')" 2>&1 | grep Exception | awk '{gsub("/nonexist.txt","",$9); print $9}') +WORKING_DIR="${USER_FILES_PATH}/${CLICKHOUSE_TEST_UNIQUE_NAME}" +mkdir -p "${WORKING_DIR}" +DATA_FILE="${WORKING_DIR}/all_null.parquet" + +# 1000 rows, 100 rows per row group => 10 row groups. `k` varies; `c_null` is NULL in every row, so +# each of its column chunks has null_count == num_values and no min/max value. The reader +# materializes such chunks directly from statistics without fetching any data pages. +${CLICKHOUSE_CLIENT} -q " + INSERT INTO FUNCTION file('${DATA_FILE}', Parquet) + SELECT + number AS k, + NULL::Nullable(Int64) AS c_null + FROM numbers(1000) + SETTINGS engine_file_truncate_on_insert = 1, output_format_parquet_row_group_size = 100 +" + +STRUCTURE="k UInt64, c_null Nullable(Int64)" + +qid_on="${CLICKHOUSE_TEST_UNIQUE_NAME}_on" +qid_off="${CLICKHOUSE_TEST_UNIQUE_NAME}_off" + +echo "-- all-null column, optimization on" +${CLICKHOUSE_CLIENT} --query_id="${qid_on}" -q " + SELECT c_null IS NULL AS is_null, count() + FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') + GROUP BY 1 +" + +echo "-- all-null column, optimization off (must be identical)" +${CLICKHOUSE_CLIENT} --query_id="${qid_off}" -q " + SELECT c_null IS NULL AS is_null, count() + FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') + GROUP BY 1 + SETTINGS input_format_parquet_use_constant_column_optimization = 0 +" + +echo "-- the varying column is read correctly (not treated as constant)" +${CLICKHOUSE_CLIENT} -q "SELECT sum(k), min(k), max(k), count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}')" + +echo "-- IS NULL / IS NOT NULL filters on an all-null column" +${CLICKHOUSE_CLIENT} -q "SELECT count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') WHERE c_null IS NULL" +${CLICKHOUSE_CLIENT} -q "SELECT count() FROM file('${DATA_FILE}', Parquet, '${STRUCTURE}') WHERE c_null IS NOT NULL" + +echo "-- null_as_default: a non-nullable hint substitutes the default (0) for every row" +${CLICKHOUSE_CLIENT} -q " + SELECT c_null, count() + FROM file('${DATA_FILE}', Parquet, 'k UInt64, c_null Int64') + GROUP BY 1 + SETTINGS input_format_null_as_default = 1 +" + +echo "-- optimization fired only when enabled" +${CLICKHOUSE_CLIENT} -q " + SYSTEM FLUSH LOGS query_log; + SELECT ProfileEvents['ParquetConstantColumnChunks'] > 0 + FROM system.query_log + WHERE event_date >= yesterday() AND event_time >= now() - 600 + AND query_id = '${qid_on}' AND type = 'QueryFinish' AND current_database = currentDatabase(); + SELECT ProfileEvents['ParquetConstantColumnChunks'] = 0 + FROM system.query_log + WHERE event_date >= yesterday() AND event_time >= now() - 600 + AND query_id = '${qid_off}' AND type = 'QueryFinish' AND current_database = currentDatabase(); +" + +rm -rf "${WORKING_DIR}"