Repository navigation
The uplink and the app session over QUIC, with 0-RTT a replay cannot use (QUIC steps 2–3) - #22
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 137f261651
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if !c.bulkIn || gotBulk { | ||
| c.closeWith(cloudwire.CloseProtocol, "") | ||
| return |
There was a problem hiding this comment.
A phone connection is constructed with bulkIn == false, so this branch closes it as soon as the phone sends a lane-1 tunnel message on its QUIC bulk stream. The new protocol advertises a bulk stream each way, but only box-to-phone bulk currently works; phone-to-box bulk traffic therefore terminates the app session instead of reaching the box, and the app handler would also need to accept and preserve wire.Bulk for tun messages.
Useful? React with 👍 / 👎.
| select { | ||
| case <-c.resumed: | ||
| continue | ||
| case <-ctx.Done(): | ||
| return |
There was a problem hiding this comment.
When the server rejects 0-RTT—for example after the ticket's early-data window expires—the original QUIC connection context is canceled with Err0RTTRejected, so this select can return on ctx.Done() instead of waiting for afterEarly to install and replay onto NextConnection. The control reader then exits without reporting an error, leaving the resumed app session unable to receive its response and eventually timing out; wait for the replacement signal and bind subsequent readers to the replacement connection rather than the original context.
Useful? React with 👍 / 👎.
quicdoor serves h3 with the TCP handler and certificate, Alt-Svc points clients at it, ftwcloudd health checks both, and compose maps 443/udp. A stateless reset key in the state directory lets the next process reset an old connection after a restart. Past 200 new connections a second the door asks for a Retry, a handshake gets 5 s, and a UDP port that will not bind leaves TCP serving alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
quicdoor takes ftw-cloud-box/1 and ftw-cloud-app/1 beside h3 and hands each connection to the WebSocket's handler through wire.Conn: a control stream that starts with the target, a bulk stream each way for lane 1 tun, and DATAGRAMs for hb and a table-less live. wsclose tracks QUIC connections too, so the drain closes them with 4503. store.TunOut carries Bulk. Door.Close ends live sessions with 1001 on the way out. The go/protocol pin moves to srcfl/ftw#1653. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- B1: a phone's session waits for the QUIC handshake, the first copy of an early flight too, so a flight the network dropped and replays later replaces nothing, spends no pairing code and asks for no watch. The ephemeral-key guard is gone; EarlyHeld counts sessions that waited. - B2: the door offers only the ALPNs it serves, so with the sessions off a client's handshake for them fails at once. - S2: a late live whose srcRev is above the table held still gives its table (store.PutTable), and the newest values stay. - S4: flow-control windows of 256 KB a stream and 512 KB a connection, at most 20,000 connections and 1,000 handshakes at once. A full door closes after the handshake with 4429, so the client reads the code. - Tests: a dropped early flight replayed after the phone dialled again, a door without the sessions, an early box target's exporter, a full door, a late table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f2ebdf5 to
c607a18
Compare
…h main merged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 participant
QUIC, steps 2 and 3, the service's side:
/v1/boxand/v1/appover QUIC on UDP 443, with the WebSockets' handlers, and 0-RTT that a replay cannot use. It is stacked on #16 (HTTP/3) and paired with srcfl/ftw#1653 (protocol and box) and srcfl/ftw-app#74 (iPhone). Both are rebased ontomain, with #17 (scale) and #21 (MCP) in, and carry the review's fixes.go/protocolis pinned to srcfl/ftwc49da79donfeat/cloud-quic; I move the pin to the merge commit once srcfl/ftw#1653 lands.What changes
internal/quicdoornow also takesftw-cloud-box/1andftw-cloud-app/1on the same UDP port as HTTP/3, and offers only the ALPNs it serves.Conncarries the WebSockets' messages:hband a table-lessliveas datagrams.QUIC's keepalive and idle timeout stand in for the ping. Close codes are the QUIC application error. A second stream of either kind closes the connection with 4400.
internal/wireis the WebSocket's method set, which a*websocket.Connalready has. The hooks in the packages cloud-scale owns are small:uplink.ServeConnandappsession.ServeConn, which the WebSockets' handlers call after the upgrade, so feat: a seam for shards, a quicker hot path, metrics, deploys without a storm, and a load harness #17's checks afterhellocover QUIC too;conn wire.Connin place of*websocket.Conn, andwsclose.Socketholds awire.Conn, so the drain closes QUIC sessions with 4503 as well;store.TunOutcarriesBulk, so a bulktunfrom the box goes on the phone's bulk stream.The uplink closes on a datagram that is not newest-wins, and on anything but
tunon the bulk stream. It drops anhborlivewhoseseqis not above the newest of its class on the connection, except that a latelivewith a newer table still gives its table (store.PutTable), and the newest values stay.A phone gets everything on the control stream until its first transport message shows that it holds message 2.
0-RTT.
WrapSession).watch(EarlyHeldcounts sessions that waited).challengemay go back before the handshake ends.Limits. Flow-control windows of 256 KB a stream and 512 KB a connection. At most 20,000 connections and 1,000 handshakes at once: a full door closes a newcomer after its handshake with 4429, so the client reads the code; too many handshakes are shed at once, and those clients take the WebSocket. From HTTP/3 for the plain HTTPS routes, on UDP 443 beside TCP (QUIC step 1) #16: a Retry past 200 new connections a second, and a 5 s handshake.
-quic-sessions(andFTW_CLOUD_QUIC_SESSIONSin compose) turns the sessions off without a release, beside-h3. With it off the door no longer offers the session ALPNs, so a client's handshake fails at once and it takes the WebSocket.deploy/http3.mdcovers both. The security group change from HTTP/3 for the plain HTTPS routes, on UDP 443 beside TCP (QUIC step 1) #16 is all the sessions need.Restarts. SIGTERM only stops the door taking connections. feat: a seam for shards, a quicker hot path, metrics, deploys without a storm, and a load harness #17's drain closes the live sessions with 4503 and a spread, and
Door.Close, deferred inmain, then ends any still open with 1001 and closes the transport. From HTTP/3 for the plain HTTPS routes, on UDP 443 beside TCP (QUIC step 1) #16, a stateless reset key in/statecovers a crash. A UDP port that will not bind leaves TCP serving.The tests that show it
TestACommandCrossesQUIC: a command over QUIC on both legs, and over QUIC on one leg with the WebSocket on the other.TestBulkTravelsOnTheBulkStream,TestHbAndLiveTravelAsDatagrams(with "nothing but the stream before the phone speaks"),TestALateDatagramIsDropped,TestALateTableStillCounts.TestHelloOverQUICIsBoundToItsTLSSession: hello carries the exporter of the TLS session inside QUIC; one from another session gets nohello_ok.TestABoxsEarlyTargetGetsItsSessionsExporter: the exporter the door hands the box's handler for an early target is the client's.TestARefusalOverQUICCarriesItsCode: 4403 and 4400 cross as application errors.TestThePhoneSessionSurvivesANetworkSwitch: mid-session the phone moves to a new UDP socket, a new source address, by probing and switching the path. The next command gets its ack and result. The box runs no new handshake, and the local address is the new one.TestTheDoorOffersOnlyWhatItServes: a door without the sessions refuses their ALPN in the handshake, within a second.TestAFullDoorSaysItIsBusy: past the limit a newcomer reads 4429.TestCloseEndsEachSessionWithGoingAway: after the context ends, the door takes no new connection and the live session still carries messages;Closethen ends it with 1001.TestADroppedEarlyFlightReplayedLaterChangesNothing:watchand notun; the live session is not replaced and its next command works; the pairing code still admits the phone that holds it.Without the wait for the handshake the test fails: the replay reaches the box.
Numbers: today's WebSocket against QUIC
A real
ftwcloudd, a box running Core's owncloudlink,appuplinkand command handler, and a test phone in Go (go/protocolandquicdoor.Dial) that pairs with it and speaks the app protocol inside the tunnel. The WebSocket rows are today's path on both legs; the QUIC rows are QUIC on both. Everything runs in one Linux container whose loopback carries netem: 20 ms each way, so every leg has a 40 ms round trip, and the loss applies to each packet on both legs. The readable copy is on. 30 connects and 30 taps per row, 90 s of frames.The connect rows come from a run with the review's fixes in, where a phone's session waits for the QUIC handshake. The other rows come from the run before the fixes, which do not touch a session once it is open.
What it says: QUIC saves round trips where a connection starts, and it keeps loss on one stream from holding up the others. Cold, it is two round trips less: the TCP handshake and the WebSocket upgrade. A phone's reconnect now costs what a cold connect does, since its session waits for the handshake; 0-RTT still saves a box a round trip, whose
challengecomes back in the first flight. Under loss the tails shrink everywhere, and a network switch costs no new session. With no loss, a frame or a tap takes one round trip either way.The copy's
liveat 5 % loss is not a clean win. QUIC sends it as a datagram, never again, and lost 8 of the 90; the WebSocket lost none but brought its late ones up to 309 ms after. A lost one shows as a gap in the freshness rules (hb.liveSeq), never as a live value. Neither row measures the age of the newest value at the phone, which is what a person sees.now.tun), the box session's handshake andhellowith its subscription, to the snapshot.cmd.ack.live, from the box'swallMsto the phone. Over QUIC it goes as a datagram; the counts say how many of the 90 the box sent arrived.On the phone: FTWKit in the iOS simulator
LiveQUICTests(in srcfl/ftw-app#74) runs the app's own stack (AppModel,PairModel,CloudCarrier, the tunnel and the session) against a realftwclouddand a real Core box with the relay unreachable. It runs in an iPhone 17 simulator. The service and the box run in a container whose netem shapes only the phone's leg, 20 ms each way plus OrbStack's own few ms (ping 56–100 ms); the box reaches the cloud unshaped. The readable copy is off. Each row is one pairing, 30 taps and 6 returns from the background.The Mac's own
swift testgave the same at 40 ms: 305 against 180 ms from scan to stream, and 269 against 184 ms back from the background. In an earlier 5 % run the race gave one session to the WebSocket, because one lost QUIC packet made QUIC miss its 250 ms lead. A QUIC dial that loses now goes on to learn, so that one loss no longer keeps QUIC second for a day.Keeping an idle box linked
The box uplink stayed idle for 180 s with the readable copy off, at 40 ms, counted packet by packet on the loopback (IP bytes):
In the first two minutes, both ends also send path MTU probes of about 1.4 KB beside each keepalive. Over the 180 s window that came to 10 KB, a rate of 4.9 MB a day that stops once discovery ends. A PING every 15 s keeps a 30 s NAT mapping open.
Not checked
QUIC.TLShas no ticket switch; the olderNWProtocolQUIC.Optionsdoes. A phone gains nothing from 0-RTT now anyway: its session waits for the handshake.The rig is not in any repository. It is the box as a test binary in Core's
appuplink(with Core's own stubs), a phone in Go overgo/protocolandquicdoor.Dial,run.sh, and netem in an Alpine container on OrbStack. Landing it as a check is a follow-up.Checks
go vet ./...,go test -race ./...,go test -tags ftwclouddev ./...,scripts/check-imports.sh: pass on a laptop after the rebase and the fixes.PutTable.Size
About 1,150 lines of service Go beside #16, about 1,450 with it.
quicdooris 1,080 lines in all; about 150 of them are the client side that tests and measurements dial with.wireis 65, and the hooks are the rest. The service was past its 3,000-line budget before this.🤖 Generated with Claude Code