Performance issue using abortableSource()

[twoeths]

• Version:
  0.39.2

• Platform:
  Linux 5.15.0-25-generic Update libp2p-spdy to version 0.9.0 🚀 #25-Ubuntu SMP Wed Mar 30 15:54:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

• Subsystem:

Severity:

High

Description:

The abortableSource() api is used across libp2p stacks and it takes more than 7.5% of lodestar cpu time, which causes the I/O lag issue in lodestar. I think this is too much considering this is not a core feature when running a node in lodestar.

yarn why v1.22.17
[1/4] 🤔  Why do we have the module "abortable-iterator"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "abortable-iterator@4.0.2"
info Reasons this module exists
   - "_project_#@lodestar#beacon-node#@chainsafe#libp2p-gossipsub" depends on it
   - Hoisted from "_project_#@lodestar#beacon-node#@chainsafe#libp2p-gossipsub#abortable-iterator"
   - Hoisted from "_project_#@lodestar#beacon-node#@libp2p#mplex#abortable-iterator"
   - Hoisted from "_project_#@lodestar#beacon-node#@libp2p#tcp#abortable-iterator"
   - Hoisted from "_project_#@libp2p#peer-record#@libp2p#utils#abortable-iterator"
   - Hoisted from "_project_#@lodestar#beacon-node#libp2p#abortable-iterator"
   - Hoisted from "_project_#@lodestar#beacon-node#libp2p#@libp2p#multistream-select#abortable-iterator"
   - Hoisted from "_project_#@lodestar#beacon-node#@chainsafe#libp2p-gossipsub#@libp2p#pubsub#abortable-iterator"
info Disk size without dependencies: "120KB"
info Disk size with unique dependencies: "184KB"
info Disk size with transitive dependencies: "184KB"
info Number of shared dependencies: 2
✨  Done in 0.96s.

I suggest we need to find a more efficient abortable pattern and apply it everywhere

cc @wemeetagain @mpetrunic @dapplion

Steps to reproduce the error:

Run lodestar, monitor metrics and take a profile from there

[twoeths]

attaching the detail of time used in aboratableSource() function of abortable-iterator package

[dapplion]

A core value in Rust is "zero cost abstractions". As Javascript developers we must be mindful that in this language this is not the case. I worry that in the current libp2p implementation a single package requires a very high amount of iterator cycles, just for a single package.

More investigation is needed to understand the costs, but on a preliminary review it's a apparent that the amount of Promises and related objects per packet may be excessive.