Skip to content

feat: serve traffic reports from the sync operator app - #103

Closed
salindne wants to merge 19 commits into
mainfrom
feat/sync-operator-traffic-report
Closed

salindne wants to merge 19 commits into
mainfrom
feat/sync-operator-traffic-report

Conversation

@salindne

@salindne salindne commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Refs ChainSafe/canton-extending-mainnet#152

Summary: the sync operator app reports the traffic charged to participants per 10-minute interval, summed across participants as #87 asks. Once an interval is over, TrafficReportTrigger reads every member's consumed traffic from the sequencer's GetLsuTrafficControlState at its start and end and stores the participants' total, since the sequencer prunes its journal. GET /v0/traffic-report serves each interval as Ok, Undetermined or CannotProvide. It needs a JWT, so auth is a new required config field.

A successor sequencer takes over the counters at an upgrade but not their history, so it reports zero for every member before the upgrade time. The trigger reads up to the announced upgrade time from the current sequencer and later times from the successor. Once the successor is current, an interval spanning the upgrade that wasn't stored yet reports CannotProvide.

Both queries use the primary key, here with a year of intervals (52,560 rows):

Query plans
lookupLatestIntervalStart: select max(interval_start) from sync_operator_traffic_report
 Result  (cost=0.32..0.33 rows=1 width=8) (actual time=0.009..0.009 rows=1.00 loops=1)
   InitPlan 1
     ->  Limit  (cost=0.29..0.32 rows=1 width=8) (actual time=0.007..0.007 rows=1.00 loops=1)
           ->  Index Only Scan Backward using sync_operator_traffic_report_pkey on sync_operator_traffic_report  (cost=0.29..1660.69 rows=52560 width=8) (actual time=0.006..0.006 rows=1.00 loops=1)
                 Heap Fetches: 1
 Execution Time: 0.017 ms

listIntervals, a full page of 1000 intervals
 Index Scan using sync_operator_traffic_report_pkey on sync_operator_traffic_report  (cost=0.29..41.77 rows=1024 width=16) (actual time=0.005..0.063 rows=1000.00 loops=1)
   Index Cond: ((interval_start >= '1779225600000000'::bigint) AND (interval_start < '1779825600000000'::bigint))
 Execution Time: 0.086 ms
  • GetLsuTrafficControlState with a timestamp is documented for disaster recovery; is calling it every 10 minutes OK?
  • Paymasters build their own automation against this endpoint (Tracking: DSync Paymaster Support in Splice 0.11.0 #87), but the spec is sync-operator-internal.yaml. Should it be external?

sadiq1971 and others added 13 commits November 4, 2026 02:52
Ports ReconcileDedicatedSynchronizerParametersTrigger from archive/prepaid-traffic-main at
086bea6 (originally #33, #65): a zero base traffic rate and the global synchronizer's read
cost, read through Scan. The Scan client comes back with it. The parameter checks from the
archive's SyncOperatorTrafficIntegrationTest move into SyncOperatorIntegrationTest.

Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
…sync operator

Ported from archive/prepaid-traffic-main at 086bea6 (originally part of #31): the task
type, the grant and the staleness check of SvOnboardingUnlimitedTrafficTrigger move to
GrantUnlimitedTrafficTriggerBase in apps-common, so the sync operator's triggers can reuse
them. Staleness uses lookupSequencerTrafficControlState, so a member whose traffic state
has gone is treated as stale instead of failing the task.

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
…nchronizer [ci]

The dedicated synchronizer runs at a zero base rate and the paymaster pays afterwards, so
the sync operator keeps every member's traffic limit at the maximum:

- MediatorUnlimitedTrafficTrigger, ported from archive/prepaid-traffic-main at 086bea6
  (originally #31), for the synchronizer's mediators.
- ParticipantUnlimitedTrafficTrigger for participants that have a
  ParticipantSynchronizerPermission and have joined. A participant without a permission
  gets nothing, even on an open synchronizer.
- PreJoinParticipantTrafficTrigger buys the maximum for admitted participants before they
  join, so a late joiner can finish connecting. Canton accepts this from protocol version
  37; below that the trigger does nothing and the grant follows the join.

The sync operator tests run splitwell on protocol version 37 (start-canton.sh -t).
SyncOperatorIntegrationTest checks the mediator grant, that a joined participant without a
permission gets nothing, that admitting it grants the maximum, and that a participant
admitted before it joins connects with the maximum and transacts.

Closes ChainSafe/canton-extending-mainnet#151

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
A ping from bob to alice over splitwell also waits for alice's vetting of the ping package to
reach splitwell. Alice's topology was refused for lack of traffic until she was admitted, and
her outbox had not caught up within the test's window in CI. Bob's connect only returns once
his own topology is sequenced, so a ping to himself checks that he transacts without
depending on alice.

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
Signed-off-by: sadiq1971 <sadiqurr8@gmail.com>
…nlimited-traffic [ci]

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
The sync operator stores the traffic charged to participants in each finished 10-minute interval, read from the sequencer at both ends, and serves it on an authenticated GET /v0/traffic-report.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
…sync operator

Ported from archive/prepaid-traffic-main at 086bea6 (originally part of #31): the task
type, the grant and the staleness check of SvOnboardingUnlimitedTrafficTrigger move to
GrantUnlimitedTrafficTriggerBase in apps-common, so the sync operator's triggers can reuse
them. Staleness uses lookupSequencerTrafficControlState, so a member whose traffic state
has gone is treated as stale instead of failing the task.

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
…nchronizer [ci]

The dedicated synchronizer runs at a zero base rate and the paymaster pays afterwards, so
the sync operator keeps every member's traffic limit at the maximum:

- MediatorUnlimitedTrafficTrigger, ported from archive/prepaid-traffic-main at 086bea6
  (originally #31), for the synchronizer's mediators.
- ParticipantUnlimitedTrafficTrigger for participants that have a
  ParticipantSynchronizerPermission and have joined. A participant without a permission
  gets nothing, even on an open synchronizer.
- PreJoinParticipantTrafficTrigger buys the maximum for admitted participants before they
  join, so a late joiner can finish connecting. Canton accepts this from protocol version
  37; below that the trigger does nothing and the grant follows the join.

The sync operator tests run splitwell on protocol version 37 (start-canton.sh -t).
SyncOperatorIntegrationTest checks the mediator grant, that a joined participant without a
permission gets nothing, that admitting it grants the maximum, and that a participant
admitted before it joins connects with the maximum and transacts.

Closes ChainSafe/canton-extending-mainnet#151

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
A ping from bob to alice over splitwell also waits for alice's vetting of the ping package to
reach splitwell. Alice's topology was refused for lack of traffic until she was admitted, and
her outbox had not caught up within the test's window in CI. Bob's connect only returns once
his own topology is sequenced, so a ping to himself checks that he transacts without
depending on alice.

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
The participant admin API in integration tests now only allows the
services Splice uses, which excludes the ping service.

Signed-off-by: Haiko Schol <haiko@chainsafe.io>
@haikoschol
haikoschol force-pushed the feat/sync-operator-unlimited-traffic branch from 1dd309c to a5d8ecf Compare November 9, 2026 11:40
…erator-traffic-report

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
…affic-report

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
A successor sequencer takes over the counters at the upgrade but not their history, so it
reports zero before it. Read up to the announced upgrade time from the current sequencer and
later times from the successor, and once the successor is current, don't read it from before
its upgrade.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
Fork main took V082.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
The app serves one logical synchronizer, which keeps its id across an upgrade, so the key
needs only the interval. Also say why an interval can be stored twice, drop a store test that
the trigger tests cover, and lowercase the integration test's clue labels like the rest of the
file.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
@salindne
salindne changed the base branch from feat/sync-operator-unlimited-traffic to main November 13, 2026 19:05
@salindne salindne closed this Nov 13, 2026
@salindne salindne reopened this Nov 13, 2026
@salindne

Copy link
Copy Markdown
Collaborator Author

Replaced by #111

@salindne salindne closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants