Repository navigation
fix(remote): two daemons, phone tabs, a split witness, and pairing clock skew - #488
Merged
Merged
Conversation
The gateway writes each step to all three replicas and the endpoint needs all three receipts. A control plane that stopped part way left some replicas one step ahead, and lineage recovery then refused that lineage forever, so its pairing never worked again until a re-pair. Before recovery, the gateway now brings the replicas left behind forward by the same endpoint-signed step. A replica takes it only for a lineage it has not recovered, only as a direct successor of its own head, and only when fresh heads both other replicas signed for one endpoint-signed read sit at its head or that successor. A missed registration completes as an ordinary registration. Heads for the comparison come from currentHead, which does not keep the answer as the read's response, so the same read sees the new head afterwards. No replica moves backward and every head is one the endpoint signed: an endpoint restored from before its pending step is still caught as a fork, and replicas holding two different successors stay unrecovered. Signed-off-by: Lokesh <lokeshselvam7025@gmail.com>
Two daemons for one user (axl and axl --unsafe keep separate state) share the installation's relay identity. With /remote in both, each replaced the other's relay route forever: both phones hung without an error and each daemon received the other phone's requests. The daemon serving remote access now holds a claim file under ~/.axl/remote, and another daemon's /remote is refused with its process and state directory before its own pairing is touched; its restore waits. On the phone, the tab opened last always wins. A tab listens for takeovers from the start, a new tab repeats its request while it waits and then takes the endpoint lock from a tab that never answers (a suspended background tab) through the binding's new takeOverDeviceEndpoint, and the owner is marked in localStorage so a tab that wakes later stops instead of reconnecting over the new one. A tab that let go keeps saying why. A request that overtakes the pairing activation is dropped quietly instead of being recorded as a failure; the phone resends both. The phone E2E gains three scenarios: tabs opened back to back, a frozen tab with no tab channel, and a second daemon's refused /remote. Signed-off-by: Lokesh <lokeshselvam7025@gmail.com>
A daemon refused the KeyPackage of a phone whose clock was even a second ahead of its own (OpenMLS checks that the lifetime has begun), and a phone refused an invitation issued after its own clock read. WSL clocks drift after sleep, so real pairings could fail with an opaque error. The same check was the intermittent join failure in the Chrome and Safari barrier jobs: the test peer's clock is captured before the device creates its KeyPackage. The device now starts its KeyPackage lifetime two minutes early, as OpenMLS's default lifetimes do, and still ends it ten minutes from now; the daemon accepts that span and the exact ten-minute span of older devices. The device accepts an invitation issued up to two minutes ahead of its clock. Each side's checks against its own clock stay exact. The barrier scenario runs its peer 1.5 s behind on purpose. Signed-off-by: Lokesh <lokeshselvam7025@gmail.com>
…pairing skew The hosted-path checkpoint now records how each of its gates was settled instead of listing them as open. Signed-off-by: Lokesh <lokeshselvam7025@gmail.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
A tab that took the endpoint lock from a tab that never let go tried to open the endpoint once more and gave up if the lock still looked held. The lock manager can see the taken lock's release after the worker's next request, and on a slow runner the new tab then said the pairing was open elsewhere. The tab now keeps trying for the handoff window after the takeover, taking the lock again every few attempts. Signed-off-by: Lokesh <lokeshselvam7025@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the edge cases found while testing remote control with several daemons, several phone tabs, and restarts, plus two known bugs.
Two daemons running
/remote(the "later tabs silently stop working" report)Every daemon a user runs (
axl,axl --unsafe, another sandbox) has its own state directory and pairing, but they share the installation and so its relay identity. With/remotein two of them, each connection replaced the other's relay route forever: the reproduction showed both relays stuck inreconnecting, the first phone stuck on "Catching up", the second on "Opening session", no error anywhere, and the second daemon receiving the first phone's requests (Unexpected application_request before pairing completed, thenidentity_mismatch).Now one daemon at a time serves remote access. It holds a claim file in
~/.axl/remote/<installation>.lock(process and state directory, taken over when the owner has exited). Another daemon's/remotefails with "Remote access is in use by another Axl daemon (process N, state in ...)", before its current pairing is touched; its restore waits for the claim.Phone tabs
The newest tab always wins, deterministically:
takeOverDeviceEndpointin the browser binding; that tab's endpoint fails closed exactly as for any lost lock;Witness split by a crash
The gateway writes each step to all three replicas, and the endpoint needs all three receipts. A crash part way left some replicas a step ahead, and recovery refused that lineage forever, so the pairing was dead until a re-pair. Before recovery the gateway now brings the replicas left behind forward by the same endpoint-signed step (
catchUpLineage), only for an unrecovered lineage, only as a direct successor of the replica's own head, and only when fresh heads both other replicas signed for one endpoint-signed read sit at that head or that successor. A missed registration completes as an ordinary registration. No replica moves backward; an endpoint restored from before its pending step is still caught as a fork, and replicas holding two different successors stay unrecovered.Browser pairing flake, and a real clock-skew bug behind it
The intermittent
internal_error: joinin the Chrome and Safari barrier jobs (6 of the last 100 CI runs) was a KeyPackage lifetime check: the test peer's clock is captured before the device creates its KeyPackage, and when a second boundary fell between them the KeyPackage looked not yet valid. The same strictness affects real pairing: a phone whose clock is even a second ahead of the daemon (WSL clocks drift after sleep) had its KeyPackage refused by OpenMLS, and a phone behind the daemon refused the invitation. The device now starts its KeyPackage lifetime two minutes early (as OpenMLS's default lifetimes do) and accepts an invitation issued up to two minutes ahead of its clock. The daemon accepts the new 12-minute span and the old 10-minute one. Each side's checks against its own clock stay exact. The barrier scenario now runs its peer 1.5 s behind on purpose.Smaller
remote-hosted-path.mdlisted gates that are long settled; it now records how each was settled.Tests
/remote./remote.