[lhoward]

Summary

Cancelling SocketPool.run() while the pool waits on its event queue throws the interrupted wait's error instead of CancellationError, and HTTPServer.run() then logs it as a critical server error. The result is that every cancelled HTTPServer on Darwin logs:

server error: ... kqueue kevent ... Bad file descriptor

Cause

getNotifications() runs the blocking wait on its dispatch queue, and its cancellation handler calls stopQueue() to break that wait. For kQueue, stop() closes the kqueue while kevent is still waiting on it. So kevent returns EBADF, getNotifications() throws SocketError.makeFailed("kqueue kevent"), and that becomes run()'s error.

Changes

• SocketPool.getNotifications(): if the wait fails after the task has been cancelled, throw CancellationError.
• HTTPServer.run(): don't log a CancellationError as a critical server error. Other errors are still logged.

On Linux, as far as I can tell from reading the code, the ePoll queue already comes out as CancellationError: stop() wakes the wait through the canary eventfd, and the next getNotifications() call's Task.checkCancellation() throws. So only the logging change applies there.

Reproducing

It's a race, and whether it happens depends on how far the pool has got when it's cancelled. The existing taskCanBeCancelled cancels as soon as the server is listening, before the pool is waiting on its queue, so it doesn't see this. I cancelled a server 200 ms after it started listening, three times per build:

build	runs ending in SocketError (kqueue kevent, EBADF)
0.26.2	3 of 3
0.27.1	2 of 3
main (bd32d43)	2 of 3

Tests

• New: HTTPServerTests.taskCanBeCancelled_WhileWaitingForConnections. It does that 5 times and expects CancellationError each time.
  • Without the SocketPool change it failed every time I ran it, with 5 or 6 issues per run, each expected error of type CancellationError, but ".failed(type: "kqueue kevent", errno: 9, ...)" of type SocketError was thrown instead.
  • With the change it passes.

The full suite passes on macOS (Swift 6.3.3). In 5 runs, one hung in HTTPClientTests.client_sends_request(), which never cancels a server while it runs. Unmodified main hangs the same way, in 1 of 6 runs, there in handlerError_ResponseIncludesDateHeader(). So that's an existing intermittent hang, not something this change introduces. I haven't run the suite on Linux.

We found this in SwiftOCA, whose device endpoints use SocketPool the same way HTTPServer does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Fq7ie1LzJERrh9uRuZZWE

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.77%. Comparing base (e8332f9) to head (2d01eb6).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #242   +/-   ##
=======================================
  Coverage   93.76%   93.77%           
=======================================
  Files          72       72           
  Lines        3866     3872    +6     
=======================================
+ Hits         3625     3631    +6     
  Misses        241      241           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[lhoward]

The Linux Swift 6.0 and 6.2 failures here are in sendMessage_WithPacketInfo_RoundTripsDatagram, and they aren't caused by this change. withPacketInfoControl never sets ipi_spec_dst, so the IPv4 source address sendmsg uses is whatever happens to be in memory, and it fails with ENETUNREACH on some builds but not others. The tests this PR adds pass on those same jobs. #243 fixes the pktinfo bug; once it's merged, re-running the jobs here should pass.

[lhoward]

Rebased on main. The #243 commit has dropped out, since it landed via #247, and so has the change making taskCanBeCancelled expect CancellationError, which main already does. This PR is now one commit.

A heads-up in case CI trips on it: since #252, HTTPServerTests.restarts_AfterStopped is flaky on main. Run on its own on macOS, it failed 17 of 40 times on main and 3 of 40 with this PR. stop() with the default timeout of 0 cancels the server task right after closing the listening socket. The pool child then races the accept loop's SocketError.disconnected, mostly failing with kqueue add kevent / EBADF from registering a socket after the queue has stopped. This PR doesn't touch that path; it only turns the cancelled wait's kevent failure into CancellationError.

[swhitty]

Thanks, this is a good catch, and the write-up made it easy to follow. I merged it onto current main locally and checked it. Without the SocketPool change, taskCanBeCancelled_WhileWaitingForConnections fails with kqueue kevent errno 9; with it, the test passes. The full suite passes on macOS, and CI is green everywhere.

Two small changes before merging:

1. In HTTPServer.run(), decide whether to log based on cancellation, not the error's type.
SocketPool.cancelAll() resumes every waiter with CancellationError, including the listener's accept, whenever run() exits, even after a real failure. In start(on:), group.next() returns whichever child finishes first. So a genuine pool error can reach run() as a CancellationError and is no longer logged. Checking Task.isCancelled covers the case this PR is about:

} catch {
    if !Task.isCancelled {
        logger.logCritical("server error: \(error.localizedDescription)")
    }

2. Linux is affected too. Could you update the comment and description?
ePoll.stop() writes to the canary eventfd but clears canary only on the actor's copy of the queue. The copy the dispatch block is waiting on still has the valid canary, so the wait wakes, makeNotification matches the canary, and it throws SocketError.disconnected. Before this change Linux logged server error: disconnected on cancellation, so the SocketPool change fixes Linux too. The comment in getNotifications() should just say the stopped queue's wait fails, without naming EBADF/kevent specifically.

Not blocking: the new test adds about 1s (5 × 200 ms). A SocketPoolTests case with a mock EventQueue whose getNotifications() blocks until stop() and then throws would test this deterministically on every platform. Happy to take that as is, though.

For the release notes: there's no public API change, but when cancelled while waiting, SocketPool.run() and HTTPServer.run() now throw CancellationError instead of SocketError.

Separately, and outside this PR: the underlying issue is that kQueue.stop() closes the descriptor while kevent may still be using it. The dispatch block holds a stale copy of the descriptor number, so it can be reused (e.g. by another pool's kqueue()) before kevent runs. Waking the wait with EVFILT_USER / NOTE_TRIGGER instead, as ePoll does with its canary, and closing only in close(), would remove that. That can be a follow-up.