feat(storage): add a static delay open request hedging strategy

[ajayky-os]

Implement TTFB Speculative Hedging with Configurable Connect Timeouts

Overview

This PR introduces a concurrent, speculative hedging architecture to the GCS C++ SDK. The primary goal is to mitigate extreme tail latencies (e.g., 20s+ stalls) caused by OS-level TCP/kernel drops during periods of high-throughput network congestion.

This architecture introduces two primary mitigation layers:

1. Time-To-First-Byte (TTFB) Hedging: Races a secondary socket if the initial HTTP `Open()` phase stalls beyond a configurable delay.
2. libcurl Native Connection Timeout in milli seconds: HttpConnectTimeoutOption (maps to CURLOPT_CONNECTTIMEOUT_MS for downloads) is added as a complementary fail-safe against black-holed TCP connects. It is applied after the stall-timeout options because libcurl treats CURLOPT_CONNECTTIMEOUT and CURLOPT_CONNECTTIMEOUT_MS as the same setting (last writer wins).

This is the first PR in a series. A follow-up adds a dynamic strategy that adapts the hedge delay to observed latency percentiles; this PR uses a fixed, configurable delay.

Architectural Highlights

1. TTFB-Exclusive Hedging (No Data Corruption)

Naive hedging of an `ObjectReadSource` stream risks severe data corruption and network exhaustion by duplicating many payload downloads.
This implementation explicitly restricts hedging to the stream's Open Phase (TTFB). Once a socket wins the initial connection race, the background thread gracefully exits, and all subsequent payload chunk reads continue sequentially on the caller's thread. This guarantees structural integrity and prevents multi-stream bandwidth DDoS.

2. Bounded Hedging Thread Pool

To prevent queue starvation and CPU exhaustion under heavy load, the `HedgingThreadPool` enforces strict gating mechanisms:

• Instantaneous Concurrency Limiter: Bounded via `MaxConcurrentHedgesOption`.
• QPS Token Bucket: Bounded via `ReadHedgeRateLimitOption`.
• Dynamic Fallback: If `ConnectionPoolSizeOption == 0` (unpooled), the pool correctly scales to `std::thread::hardware_concurrency() * 4` to match physical network capabilities.

3. Configurable Native Connect Timeout

The existing `DownloadStallTimeoutOption` maps to `CURLOPT_LOW_SPEED_TIME`, meaning aggressive timeouts would unintentionally kill healthy large-payload streams during minor jitter.
This PR introduces `HttpConnectTimeoutOption`, which maps explicitly to `CURLOPT_CONNECTTIMEOUT_MS`, allowing users to build a strict guillotine specifically for stalled TCP handshakes without threatening payload integrity.

Usage

Users can opt-in to the Hybrid Architecture via standard configuration options:
auto options = google::cloud::Options{}
.setgoogle::cloud::storage_experimental::EnableReadHedgingOption(true)
.setgoogle::cloud::storage_experimental::ReadHedgeDelayOption(std::chrono::milliseconds(500))
.setgoogle::cloud::storage_experimental::MaxConcurrentHedgesOption(15)
.setgoogle::cloud::storage_experimental::HttpConnectTimeoutOption(std::chrono::milliseconds(1000));
auto client = gcs::Client(options);

(Note: `hedge_pool_` is only allocated if `EnableReadHedgingOption` is true, enforcing the zero-overhead principle for non-hedging users).

Performance Benchmarks

We executed a 1-hour sequential testing suite (`us-central1`, 60 concurrent workers, 1MB payloads) comparing the baseline SDK against this architecture.

Baseline (Hedging Disabled):

• Throughput: 2,204,500 requests
• p50 Latency: 41.83 ms
• p99 Latency: 165.52 ms
• Max Latency: 21,292.60 ms (Severe kernel stall)

Static Hedging (Enabled):

• Throughput: 2,347,561 requests (+143,000 throughput)
• p50 Latency: 38.27 ms
• p99 Latency: 162.20 ms
• Max Latency: 5,331.10 ms (75% absolute reduction)

[gemini-code-assist]

Code Review

This pull request introduces experimental request hedging for ReadObject() streams to reduce tail latency by racing duplicate requests. It implements HedgedObjectReadSource and a dynamically-scaling HedgingThreadPool with token-bucket rate limiting and concurrency throttling. The review comments identify critical issues that must be addressed: a potential self-join deadlock when capturing std::shared_ptr<HedgingThreadPool> in the hedge task lambda, a concurrency race condition where checking and incrementing active hedges are not atomic, and performance overhead from zero-initializing the read buffer with std::vector<char> instead of std::unique_ptr<char[]>.

[v-pratap]

HttpConnectTimeoutOption is specifically placed inside the kGet block because this PR is scoped to download hedging.

Should we change its name as well like other Options, because I think kPut an dkPost are missing this option.

[v-pratap]

Is there any limit on n? if not then what if n is very large like 100 MB and it couldn't get allocated? will it be crashed without capturing the error?

[scotthart]

Instead of adding a new Option and code that overwrites CURLOPT_CONNECTIONTIMEOUT with CURLOPT_CONNECTIONTIMEOUT_MS, what if we changed the type of TransferStallTimeoutOption to std::chrono::milliseconds, and updated all the calls to set CURLOPT_CONNECTIONTIMEOUT_MS instead of CURLOPT_CONNECTIONTIMEOUT?