Skip to content

Hand WebSocket frames and messages off without buffering - #249

Open
lhoward wants to merge 5 commits into
swhitty:mainfrom
PADL:websocket-backpressure
Open

lhoward wants to merge 5 commits into
swhitty:mainfrom
PADL:websocket-backpressure

Conversation

@lhoward

@lhoward lhoward commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

MessageFrameWSHandler.makeFrames moves frames and messages between the connection and the handler through two AsyncStream continuations with unbounded buffers:

  • incoming messages are yielded into messagesIn;
  • outgoing frames are yielded into framesOut;
  • a task pumps between them.

This has two failure modes:

  • A slow handler: a client that floods the server grows the first buffer.
  • A client that stops reading: while the handler keeps sending, the second buffer grows.

On a small ARM board that ends with the process killed. And since yield never waits, a handler can't tell when a message has reached the connection.

Change

  • AsyncHandoff<Element> (new, in FlyingSocks, package visibility) passes elements through a bounded buffer.
    • send suspends only while capacity elements are waiting; a capacity of 0 is a rendezvous.
    • finish(with:) ends it with a final element or error.
    • After finish, send throws AsyncHandoffFinishedError. CancellationError means only that the sender was cancelled.
    • A waiting sender or receiver that is cancelled is removed.
    • It is built on withIdentifiableThrowingContinuation, like SocketPool.
  • MessageFrameWSHandler uses two handoffs of capacity 16 in place of the continuations.
    • The handler's input is AsyncStream { try? await messagesIn.next() }.
    • The connection's frames come from AsyncThrowingStream { try await framesOut.next() }.
    • Back-pressure therefore reaches the client's socket in one direction and the handler's stream in the other, once 16 elements are waiting.
    • Pings and closes are still read and answered while fewer than 16 messages wait for the handler. Past that, the reader stops and TCP pushes back on the client, so a handler should keep reading its input; the README says so.
    • A handler that produces messages on demand, for example with AsyncStream(unfolding:onCancel:), is paced by the client. One that yields into a continuation keeps its own buffer, as before.
    • WSMessageHandler and WSHandler are unchanged.
  • Ending the connection: both handoffs finish as soon as either loop ends, before waiting for the other. The connection closes even if the handler's stream ignores cancellation; only the handler's own task lingers until its closure returns. When a client's stream ends without a close frame, frames already buffered are written and the connection ends without waiting for the handler.
  • AsyncStream.protocolFrames(from:onTermination:): the frames stream is now pull-based, so there is no continuation whose onTermination cancels the handler's tasks. The closure runs when the stream ends, or is released without being read to the end, and the handler cancels its tasks from there.
  • Close frames: on a client's close frame, the echo is sent and the frames finished before the .close message is delivered as the final message of the handler's stream. The closing handshake no longer waits on the handler.
  • Existing tests: responseFrames is unchanged and passes. messagesOut_Ends_WhenCloseMessageIsSent keeps its client open, since it raced the client's stream ending.

Testing

FlyingSocks: AsyncHandoffTests cover:

  • suspension when full and at capacity 0;
  • buffered delivery;
  • final element and error delivery, and the finished error;
  • cancellation of a sender or receiver.

WSHandlerTests check:

  • a ping is answered and a close echoed while the handler never reads its input;
  • the frames end when the client ends while the handler ignores cancellation;
  • the client's frame stream and the handler's stream are read no further ahead than the capacity;
  • a close frame is echoed, then delivered as the final message;
  • protocolFrames runs onTermination both when read to the end and when released early.

Results: swift test on Linux (Swift 6.4): FlyingSocks 167/167; FlyingFox 321 tests, with the same 3 issues as on main (HTTPDecoderTests.standardizedPath and naughtyPath_IsParsed). The WebSocket suites passed 300 runs in a row. Not run on Darwin locally. The branch merges cleanly with main, including #255.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CEgKfBwi5LpKqwfbctCEdH

@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.64706% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.80%. Comparing base (e845dc0) to head (a9404f7).

Files with missing lines Patch % Lines
FlyingFox/Sources/WebSocket/AsyncHandoff.swift 95.74% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #249      +/-   ##
==========================================
+ Coverage   93.76%   93.80%   +0.04%     
==========================================
  Files          72       73       +1     
  Lines        3847     3905      +58     
==========================================
+ Hits         3607     3663      +56     
- Misses        240      242       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@lhoward
lhoward force-pushed the websocket-backpressure branch from 948301b to a9404f7 Compare November 9, 2026 01:05

@swhitty swhitty left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Luke, bounding these buffers is the right goal. Unbounded AsyncStream buffers getting the process killed on a small board is a real problem, and AsyncHandoff is neat and well tested.

My main concern is that zero capacity ties the WebSocket protocol to the handler's pace:

  • Control frames stall behind data. Loop A waits in messagesIn.send until the handler takes each message, so pings and the close frame queued behind it aren't read. A handler that never iterates client (a broadcast-only server, say) stops answering pings and never echoes the close once the client has sent one message. Before, this only grew a buffer.
  • Existing code can now hang. Driving the handler by reading its messages before its frames now deadlocks; the responseFrames test had to change for exactly this. Nothing fails to compile, so users would only find out at runtime.
  • Unfolding output can outlive the connection. The README now suggests AsyncStream(unfolding:) for output. If that closure waits on app events, cancelling Loop B doesn't interrupt it, so the start task and the handler stay alive after the connection has closed.

Suggestions:

  • Keep ping and close out of the handoff, so the protocol stays responsive however slow the handler is.
  • Consider a small bounded capacity rather than zero, where send suspends only when the buffer is full. That still caps memory, which is what fixes the ARM case, and avoids a rendezvous per message and per frame (including each 16 KB fragment).
  • Add a test where a ping arrives while a data message is pending, or while the handler ignores its input.

Smaller points:

  • AsyncHandoff is a general primitive; it would sit better in FlyingSocks next to AsyncSharedReplaySequence and IdentifiableContinuation.
  • A dedicated "finished" error instead of CancellationError would let the loops tell a finished handoff apart from real cancellation.

I'm planning to land the tunnel in #254 first; this doesn't overlap with it. Either way this changes observable behaviour, so I'll release it as a minor version with notes on the above.

Review drafted with help from Claude.

if frame.opcode == .close {
throw FrameError.closed(frame)
} else if let message = try makeMessage(for: frame) {
try await messagesIn.send(message)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where the stall happens: while this send waits for the handler, the loop doesn't read the next frame. A ping or close queued behind this message gets no reply until the handler takes it, which is never if the handler doesn't iterate its input.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, this is where it stalled. In "Answer pings and closes while the WebSocket handler is behind", messagesIn and framesOut each hold up to 16 elements, so this loop goes on reading frames, and answering pings and closes, while fewer than 16 messages wait for the handler. There are new tests for a ping answered and a close echoed while the handler never reads its input. Both hang on the previous head.

@lhoward

lhoward commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks Simon. There are three new commits on the branch:

  • "Move AsyncHandoff to FlyingSocks with its own finished error"
  • "Give AsyncHandoff a bounded capacity"
  • "Answer pings and closes while the WebSocket handler is behind"

Control frames stall behind data. Fixed with a bounded capacity of 16 in place of zero. I didn't keep ping and close entirely out of the path, though. A ping arrives on the same byte stream behind the data frames before it. Answering it however slow the handler is would mean reading and holding every message ahead of it, which brings the unbounded buffer back. Now pings and closes are answered while fewer than 16 messages wait. Past that the reader stops and TCP pushes back on the client. Pongs and the close echo queue behind at most 16 outgoing frames. The README now says that a handler which doesn't need the client's messages should still read and discard them. The capacity is internal for now. I can make it an init parameter if you'd like it configurable.

Existing code can now hang. responseFrames is back in its original form and passes. Code that reads more than 16 messages before reading any frames would still wait. HTTPConnection always reads frames concurrently, though, so only something calling makeFrames directly could run into that.

Unfolding output can outlive the connection. start now finishes both handoffs as soon as either loop ends, before the task group waits for the other loop. The connection closes even if the handler's stream ignores cancellation. Only the handler's own task lingers, until its closure returns. The README now points to AsyncStream(unfolding:onCancel:) and says the producer should return when cancelled. A new test checks that frames end when the client ends while the handler ignores cancellation. One side effect: when the client's stream ends without a close frame, frames already buffered are written and the connection ends without waiting for the handler. messagesOut_Ends_WhenCloseMessageIsSent raced that, so it now keeps its client open.

Bounded capacity rather than zero. AsyncHandoff(capacity:), in "Give AsyncHandoff a bounded capacity". send suspends only while the buffer is full, and a capacity of 0 keeps the old rendezvous.

Test for a ping while a message is pending. Added: ping_IsAnswered_WhileHandlerIgnoresItsInput, plus close_IsEchoed_WhileHandlerIgnoresItsInput.

AsyncHandoff in FlyingSocks. Moved, with package visibility. Its tests are now in FlyingSocksTests.

A dedicated finished error. send now throws AsyncHandoffFinishedError once the handoff is finished. CancellationError now means only that the sender was cancelled, and the loops catch the two separately.

Landing #254 first is fine. Neither touches the other's files, and the branch still merges cleanly with main, including #255. On Linux (Swift 6.4) the tests match main: only HTTPDecoderTests.standardizedPath and naughtyPath_IsParsed fail, and they fail on main as well. A minor version with notes sounds right.

lhoward and others added 5 commits November 14, 2026 11:02
An actor that passes elements from one task to another one at a time.
send(_:) suspends until next() takes the element, so a sender gets no
further ahead than its receiver and nothing accumulates between them.
finish(with:) ends the handoff with a final element or error for the
receiver and cancels any sender still waiting; cancelling a waiting
sender or receiver removes its continuation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8cahEd4GfsHQ89sNrmP4p
MessageFrameWSHandler moved frames and messages between the connection
and the handler through two AsyncStream continuations with unbounded
buffers. A client that floods a server whose handler is slow, or that
stops reading while the handler keeps sending, grows those buffers until
the process runs out of memory; and because yielding never waits, a
handler could not tell when a message had reached the connection.

Replace the continuations with AsyncHandoffs. A client frame is now read
only when the handler has taken the previous message, and a message is
taken from the handler's stream only when the connection is about to
write the previous frame, so back-pressure reaches the client's socket
in one direction and the handler's stream in the other. A handler that
produces messages on demand, with AsyncStream(unfolding:) for instance,
is paced by the client.

The frames stream is now pull-based, so it has no continuation whose
termination cancels the handler's tasks. protocolFrames gains an
onTermination closure that runs when the stream ends or is released, and
the handler cancels its tasks from there.

On a client's close frame, the echo is sent and the frames finished
before the .close message is handed to the handler as the final message
of its stream, so the closing handshake does not wait on the handler.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8cahEd4GfsHQ89sNrmP4p
AsyncHandoff is a general primitive, not a WebSocket one, so it now
sits in FlyingSocks beside AsyncSharedReplaySequence and
IdentifiableContinuation, with package visibility, and its tests move
to FlyingSocksTests.

send(_:) threw CancellationError both when the sending task was
cancelled and when the handoff had been finished, so a loop could not
tell the two apart. It now throws AsyncHandoffFinishedError once the
handoff is finished, to a waiting sender and to one that calls it
afterwards; CancellationError means only that the sender was cancelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEgKfBwi5LpKqwfbctCEdH
With no buffer, every element was a rendezvous: the sender suspended
until the receiver had received it, once per message and once per frame,
including each fragment of a long message. A handoff can now hold up to
capacity elements, and send(_:) suspends only while that many are
waiting. Memory stays bounded, which is what matters on a small board,
without making the two tasks run in lockstep.

A capacity of 0, the default, keeps the unbuffered handoff. Once the
handoff is finished, the receiver gets the buffered elements before the
final element or error; senders still waiting for space throw
AsyncHandoffFinishedError, and a cancelled one leaves nothing behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEgKfBwi5LpKqwfbctCEdH
With unbuffered handoffs, the loop reading the client waited in
messagesIn.send(_:) until the handler had received each message, so a ping
or close frame behind that message went unread. A handler that never
iterates its input, such as one that only broadcasts, stopped answering
pings and never echoed a close once the client had sent one message.
Code that read the handler's messages before its frames, as the
responseFrames test did, deadlocked.

MessageFrameWSHandler now gives both handoffs a capacity of 16: up to
16 messages wait for the handler and up to 16 frames wait for the
connection. Frames are read, and pings answered, while fewer messages
wait; beyond that the reader stops, and the socket applies back-pressure
to the client as before. Memory stays bounded. responseFrames is back in
its original form.

When either loop ends, both handoffs are finished before the task group
waits for the other loop. A handler whose stream ignores cancellation,
such as an AsyncStream(unfolding:) closure waiting on application
events, no longer keeps the connection open after the client has gone;
only its own task outlives it. On a client's end of stream, frames
already buffered are written and the connection ends without waiting for
the handler, so messagesOut_Ends_WhenCloseMessageIsSent now keeps its
client open rather than racing its end.

The README describes the capacity, says a handler that ignores client
messages should still read them, and that an unfolding producer should
return when cancelled.

Tested: WSHandlerTests gains a ping answered and a close echoed while the
handler ignores its input, and the connection ending while the handler
ignores cancellation; each hangs without this change. The WebSocket
suites passed 200 runs in a row. swift test on Linux (Swift 6.4) fails
only HTTPDecoderTests' standardizedPath and naughtyPath_IsParsed, as on
main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEgKfBwi5LpKqwfbctCEdH
@lhoward
lhoward force-pushed the websocket-backpressure branch from 67b0ca1 to b527281 Compare November 14, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants