Skip to content

[TEST] BasicCurlHttpTests.ElegantQuitQuick is flaky: a 20 ms wall clock budget on a shared runner #4342

Description

@thc1006

BasicCurlHttpTests.ElegantQuitQuick asserts that FinishAllSessions() plus the client's destruction complete inside a 20 ms wall clock budget:

auto cost = std::chrono::system_clock::now() - beg;
ASSERT_TRUE(cost < std::chrono::milliseconds{20})

On a shared runner that budget is not something the code controls, and the test's own comment already says so ("Due to load on CI hosts (some take 10ms), we assert it is less than 20ms").

It passed and failed in the same job, on the same binaries

CMake gcc 14 (maintainer mode, async) invokes ctest twice. First pass:

100% tests passed, 0 tests failed out of 1348

Second pass, same build directory, no rebuild between them:

925/1359 Test #925: ext.http.curl.BasicCurlHttpTests.ElegantQuitQuick ... ***Failed  0.76 sec

curl_http_test.cc:712: Failure
Value of: cost < std::chrono::milliseconds{20}
  Actual: false
Expected: true
cost ms: 247 libcurl version: 0x81500

99% tests passed, 1 tests failed out of 1348

247 ms against a 20 ms budget, so it is not a marginal overshoot. The whole job then exits 8 and the PR goes red.

The bound has been 20 ms since #3198 (2024-12-18) and has not been revisited.

What could be done instead

  1. Raise the bound. Cheapest, and it keeps the shape of the check, but it only moves the threshold a loaded runner can still cross.
  2. Assert the behaviour rather than the duration. What [SDK] Do not frequently create and destroy http client threads #3198 was protecting is that the background thread wakes on shutdown instead of sitting out scheduled_delay_milliseconds_. A bound derived from that delay, say "well under scheduled_delay_milliseconds_" rather than a fixed 20 ms, states the invariant without pinning it to host speed.
  3. Keep the timing check but take it out of the default ctest set, so a slow runner does not fail an unrelated PR.

I ran into it on #4331, whose diff touches only exporters/elasticsearch. Happy to send whichever of these you prefer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/duplicateIndicates an issue is a duplicate of other open issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions