SubscribePendingTransactions returns near-zero transactions after node restart

[mukul3097]

Description

After restarting a Bor node, the eth_subscribe("newPendingTransactions") WebSocket subscription returns near-zero pending transactions instead of the expected volume. The issue does not recover over time — it persists for the lifetime of the restarted process.

Steps to Reproduce

1. Start a Bor node on Polygon mainnet and let it sync fully
2. Verify WS pending tx subscription works (should receive ~100+ tx/s)
3. Restart the Bor process (same data directory, same config)
4. Wait for the node to sync and establish peer connections
5. Test WS pending tx subscription again

Expected Behavior

After restart and peer connection, pending tx subscription should return transactions at the same rate as before restart (~100-130 tx/s on mainnet).

Actual Behavior

After restart, the subscription returns 0-2 tx/s — sometimes zero transactions for minutes at a time. The node shows:

• Synced (not in syncing state)
• Full peer count (33+ peers)
• Other RPC methods work normally (eth_blockNumber, eth_call, etc.)
• Only eth_subscribe("newPendingTransactions") is affected

Reproduction Data

Tested by subscribing to pending txs for 15-30 second windows:

Condition	tx/s received
Before restart	~130 tx/s
After restart (1 min)	0-1 tx/s
After restart (5 min)	0-1 tx/s
After restart (10 min)	0-2 tx/s
After restart (50 min)	0-1 tx/s
Non-restarted node	~130 tx/s (unchanged)

Versions Tested

• Bor v2.6.3 — same issue
• Bor v2.7.0 (commit 21a6320) — same issue

Environment

• Polygon mainnet
• Multiple deployment environments, same behavior in all
• Tested across different cloud regions

Suspected Root Cause

The SubscribeFullPendingTransactions / SubscribePendingTransactions mechanism in go-ethereum/Bor appears to not properly reinitialize the pending tx event feed after process restart. The node can receive pending txs from peers (they appear in the txpool) but the internal Go channel/event subscription that delivers them to WS subscribers is broken.

Impact

Any application relying on real-time pending transaction notifications via WebSocket will be affected after Bor node restarts, including:

• MEV/searcher infrastructure
• DEX aggregators monitoring mempool
• Transaction monitoring and analytics services

[mukul3097]

Code Analysis Update

Traced the actual server-side code path for eth_subscribe("newPendingTransactions"):

FilterAPI.NewPendingTransactions (eth/filters/api.go:212)
  → EventSystem.SubscribePendingTxs (eth/filters/filter_system.go:428)
    → eventLoop reads from es.txsCh (filter_system.go:534)
      ← txsCh fed by txsSub
        ← TxPool.SubscribeTransactions (core/txpool/txpool.go:404)
          ← LegacyPool.txFeed.Subscribe (core/txpool/legacypool/legacypool.go:537)
            ← txFeed.Send in promoteExecutables (legacypool.go:1696)

Most Likely Root Cause: eventLoop Permanent Death

The EventSystem.eventLoop goroutine exits permanently if any subscription errors:

// eth/filters/filter_system.go:556–563
case <-es.txsSub.Err():
  return // no restart, no recovery
case <-es.logsSub.Err():
  return
case <-es.rmLogsSub.Err():
  return
case <-es.chainSub.Err():
  return

txsSub is obtained via EthAPIBackend.SubscribeNewTxsEvent (eth/api_backend.go:505-507), which calls TxPool.SubscribeTransactions. This returns p.subs.Track(event.JoinSubscriptions(...)). The SubscriptionScope closes all tracked subscriptions on TxPool.Close().

If any lifecycle event causes the txpool subscriptions to close during startup (subpool reset, reorg, or JoinSubscriptions propagating a subpool error), txsSub.Err() fires and the entire eventLoop exits permanently with no restart mechanism. This explains:

• Immediate zero tx/s after every restart
• No recovery over 50+ minutes
• Reproducible across all regions and Bor versions tested

Secondary Issue: Blocking Writes in eventLoop

// eth/filters/filter_system.go:495–499
func (es *EventSystem) handleTxsEvent(filters filterIndex, ev core.NewTxsEvent) {
  for _, f := range filters[PendingTransactionsSubscription] {
    f.txs <- ev.Txs // blocking write, no select/timeout
  }
}

The eventLoop is a single goroutine. A slow WS client can block f.txs, filling the 4096-buffered txsCh, causing txFeed.Send to silently skip the subscription. This would cause gradual degradation but is less likely to produce the instant-zero behavior we observe after restart.

Notes

• SubscribeFullPendingTransactions in ethclient/gethclient/gethclient.go:206 is client-side only — the bug is entirely server-side in EventSystem.
• The reorgs parameter passed to SubscribeTransactions is silently ignored in legacypool.go:532-537 (code smell, not the primary bug).
• pendingLogsSub.Err() is notably absent from the exit conditions, unlike the other four subscriptions — an inconsistency that could mask pending logs failures.

Suggested Instrumentation to Confirm

1. Add a periodic heartbeat log in eventLoop to confirm it stops ticking post-restart
2. Log when txsSub.Err() fires in NewEventSystem to confirm Candidate 1
3. Inspect event.JoinSubscriptions in the go-ethereum event package — confirm whether a single subpool subscription error kills the entire joined subscription
4. Audit TxPool.loop (core/txpool/txpool.go:150) for early-exit conditions that call p.subs.Close() during normal startup

[mukul3097]

Are there any plans to merge the commit?

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 28 days with no activity.