Skip to content

fix(rpc): exit newPendingTransactions goroutine when event channel is closed - #4641

Open
artemrootman wants to merge 2 commits into
zeta-chain:developfrom
artemrootman:fix/pending-tx-subscription-busy-loop
Open

fix(rpc): exit newPendingTransactions goroutine when event channel is closed#4641
artemrootman wants to merge 2 commits into
zeta-chain:developfrom
artemrootman:fix/pending-tx-subscription-busy-loop

Conversation

@artemrootman

@artemrootman artemrootman commented Sep 9, 2026

Copy link
Copy Markdown

Description

pubSubAPI.subscribePendingTransactions (rpc/websockets.go) reads from the subscription event channel without checking whether the channel is still open:

case ev := <-txsCh:
    data, ok := ev.Data.(cmttypes.EventDataTx)
    if !ok {
        api.logger.Debug("event data type mismatch", "type", fmt.Sprintf("%T", ev.Data))
        continue
    }

When the client sends eth_unsubscribe or the websocket connection drops, the websockets server calls the UnsubscribeFunc returned by memEventBus.Subscribe, which closes the per-subscriber channel (txsCh) but never closes sub.Err(). A receive from a closed channel returns the zero value immediately, so the goroutine falls into a tight loop: ev.Data == nil, the type assertion fails, it formats a Debug log line and continues, forever, with no sleep. Each leaked goroutine burns a full CPU core, and since the log level is Debug nothing shows up in the logs.

subscribeNewHeads and subscribeLogs already handle this correctly with case event, ok := <-ch: if !ok { return }; this PR applies the same check to the pending-transactions handler.

Impact observed in production

On a mainnet full node (v36.0.6, 40 CPU quota) serving an RPC load balancer that opens newPendingTransactions over websocket and reconnects periodically, ~100 such goroutines accumulated over two days of uptime. The node was pinned at 100% of its CPU quota and fell behind the chain head by ~800 blocks because consensus/block execution was starved.

CPU profile (15 s, go tool pprof -top):

Duration: 15.17s, Total samples = 598.04s (3943.33%)
      flat  flat%   sum%        cum   cum%
   321.77s 53.80% 53.80%    321.77s 53.80%  runtime.futex
   ...
     7.32s  1.22% 76.58%    422.60s 70.66%  github.com/zeta-chain/node/rpc.(*pubSubAPI).subscribePendingTransactions.func1
     2.53s  0.42% 84.23%    144.93s 24.23%  fmt.Sprintf

Goroutine dump (/debug/pprof/goroutine?debug=1) showed 97 goroutines parked in subscribePendingTransactions.func1 at websockets.go:740 while only 5 websocket connections were open.

Restarting the node clears the goroutines, but they accumulate again with every client reconnect.

How Has This Been Tested?

  • go build ./rpc/ and go vet ./rpc/ pass (go1.23.8).

  • Reasoning matches the existing newHeads/logs handlers, which use the identical ok check and do not leak under the same reconnect pattern.

  • Tested CCTX in localnet

  • Tested in development environment

  • Go unit tests

  • Go integration tests

  • Tested via GitHub Actions


Note

Low Risk
Small, localized websocket subscription lifecycle fix aligned with existing handlers; reduces resource exhaustion risk without changing subscription behavior for open connections.

Overview
Fixes a goroutine and CPU leak in the newPendingTransactions websocket subscription by treating a closed event channel as a clean shutdown.

When clients unsubscribe or disconnect, the pending-tx event channel is closed but the handler kept receiving zero values and spinning in a tight loop (failed type assertions + debug logging). The change mirrors subscribeNewHeads and subscribeLogs: use ev, ok := <-txsCh and return when !ok so the goroutine exits instead of leaking.

Reviewed by Cursor Bugbot for commit 948354d. Configure here.

Greptile Summary

This PR fixes a busy-loop and goroutine leak in the pending-transactions websocket subscription handler.

  • Checks the event-channel receive status.
  • Exits the goroutine when the per-subscriber channel closes.
  • Aligns pending-transaction handling with the existing new-heads and logs handlers.

Confidence Score: 5/5

The PR appears safe to merge and correctly prevents the pending-transactions goroutine from spinning after its event channel closes.

The new receive-status check exits only after the subscription event channel has closed, while existing websocket lifecycle code continues to own unsubscription and connection cleanup; no actionable regressions were identified.

Important Files Changed

Filename Overview
rpc/websockets.go Adds the missing closed-channel check so pending-transaction subscription goroutines terminate cleanly after unsubscription or connection teardown.

Reviews (1): Last reviewed commit: "fix(rpc): exit newPendingTransactions go..." | Re-trigger Greptile

@artemrootman
artemrootman requested a review from a team as a code owner September 9, 2026 15:36

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant