lib: optimize writable stream buffer clearing #59406

ri7116 · 2025-08-08T12:46:54Z

Improved the clearBuffer function in lib/internal/streams/writable.js by replacing buffered.splice(0, i) with ArrayPrototypeSlice(buffered, i). This change eliminates the O(N) shifting overhead of splice when clearing the buffer, especially for large buffers, leading to better CPU utilization and reduced garbage collection overhead.

Profiling before and after the change showed a significant reduction in 10 CPU ticks attributed to the clearBuffer function, from 90 ticks to 1 tick.

nodejs-github-bot · 2025-08-08T12:46:59Z

Review requested:

@nodejs/streams

Improved the `clearBuffer` function by replacing `buffered.splice` with `ArrayPrototypeSlice`. - Eliminates O(N) shifting overhead. - Improves CPU utilization and reduces GC overhead.

codecov · 2025-08-08T14:23:36Z

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.91%. Comparing base (f3adc11) to head (8a8653d).
⚠️ Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/streams/writable.js	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #59406      +/-   ##
==========================================
+ Coverage   89.79%   89.91%   +0.11%     
==========================================
  Files         653      655       +2     
  Lines      192394   192829     +435     
  Branches    37650    37792     +142     
==========================================
+ Hits       172753   173374     +621     
+ Misses      12270    12005     -265     
- Partials     7371     7450      +79

Files with missing lines	Coverage Δ
lib/internal/streams/writable.js	`96.64% <0.00%> (ø)`

... and 64 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mcollina · 2025-08-08T16:52:32Z

Bench CI, streams: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1722/

Bench CI, http: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1723/

mcollina

lgtm

Ethan-Arrowood

logically makes sense. Nice catch!

nodejs-github-bot · 2025-08-08T16:55:14Z

CI: https://ci.nodejs.org/job/node-test-pull-request/68498/

gurgunday

nice!

nodejs-github-bot · 2025-08-11T03:29:02Z

Landed in 8a2fec1

Improved the `clearBuffer` function by replacing `buffered.splice` with `ArrayPrototypeSlice`. - Eliminates O(N) shifting overhead. - Improves CPU utilization and reduces GC overhead. PR-URL: #59406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Improved the `clearBuffer` function by replacing `buffered.splice` with `ArrayPrototypeSlice`. - Eliminates O(N) shifting overhead. - Improves CPU utilization and reduces GC overhead. PR-URL: nodejs#59406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [node](https://nodejs.org) ([source](https://github.com/nodejs/node)) | minor | `24.5.0` -> `24.6.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>nodejs/node (node)</summary> ### [`v24.6.0`](https://github.com/nodejs/node/releases/tag/v24.6.0): 2025-08-14, Version 24.6.0 (Current), @RafaelGSS [Compare Source](nodejs/node@v24.5.0...v24.6.0) ##### Notable Changes - \[[`471fe712b3`](nodejs/node@471fe712b3)] - **(SEMVER-MINOR)** **cli**: add NODE\_USE\_SYSTEM\_CA=1 (Joyee Cheung) [#59276](nodejs/node#59276) - \[[`38aedfbf73`](nodejs/node@38aedfbf73)] - **(SEMVER-MINOR)** **crypto**: support ML-DSA KeyObject, sign, and verify (Filip Skokan) [#59259](nodejs/node#59259) - \[[`201304537e`](nodejs/node@201304537e)] - **(SEMVER-MINOR)** **zlib**: add dictionary support to zstdCompress and zstdDecompress (lluisemper) [#59240](nodejs/node#59240) - \[[`e79c93a5d0`](nodejs/node@e79c93a5d0)] - **(SEMVER-MINOR)** **http**: add server.keepAliveTimeoutBuffer option (Haram Jeong) [#59243](nodejs/node#59243) - \[[`c144d69efc`](nodejs/node@c144d69efc)] - **lib**: docs deprecate \_http\_\* (Sebastian Beltran) [#59293](nodejs/node#59293) - \[[`aeb4de55a7`](nodejs/node@aeb4de55a7)] - **(SEMVER-MINOR)** **fs**: port SonicBoom module to fs module as Utf8Stream (James M Snell) [#58897](nodejs/node#58897) ##### Commits - \[[`f7484575ff`](nodejs/node@f7484575ff)] - **assert**: change utils to use index instead of for...of (방진혁) [#59278](nodejs/node#59278) - \[[`269cd16185`](nodejs/node@269cd16185)] - **benchmark**: remove deprecated \_extend from benchmark (Rafael Gonzaga) [#59228](nodejs/node#59228) - \[[`848e49c20b`](nodejs/node@848e49c20b)] - **benchmark**: add fs warmup to writefile-promises (Bruno Rodrigues) [#59215](nodejs/node#59215) - \[[`8c609be1b1`](nodejs/node@8c609be1b1)] - **benchmark**: add calibrate-n script (Rafael Gonzaga) [#59186](nodejs/node#59186) - \[[`6a3bf772d8`](nodejs/node@6a3bf772d8)] - **build**: fix node\_use\_sqlite for GN builds (Shelley Vohr) [#59017](nodejs/node#59017) - \[[`471fe712b3`](nodejs/node@471fe712b3)] - **(SEMVER-MINOR)** **cli**: add NODE\_USE\_SYSTEM\_CA=1 (Joyee Cheung) [#59276](nodejs/node#59276) - \[[`38aedfbf73`](nodejs/node@38aedfbf73)] - **(SEMVER-MINOR)** **crypto**: support ML-DSA KeyObject, sign, and verify (Filip Skokan) [#59259](nodejs/node#59259) - \[[`a312e706cf`](nodejs/node@a312e706cf)] - **crypto**: prepare webcrypto key import/export for modern algorithms (Filip Skokan) [#59284](nodejs/node#59284) - \[[`3a7c2c3a47`](nodejs/node@3a7c2c3a47)] - **deps**: update ada to 3.2.7 (Node.js GitHub Bot) [#59336](nodejs/node#59336) - \[[`8d9ceeaf6a`](nodejs/node@8d9ceeaf6a)] - **deps**: update archs files for openssl-3.5.2 (Node.js GitHub Bot) [#59371](nodejs/node#59371) - \[[`33b06df354`](nodejs/node@33b06df354)] - **deps**: upgrade openssl sources to openssl-3.5.2 (Node.js GitHub Bot) [#59371](nodejs/node#59371) - \[[`fa70f1af77`](nodejs/node@fa70f1af77)] - **deps**: support madvise(3C) across ALL illumos revisions (Dan McDonald) [#58237](nodejs/node#58237) - \[[`f834a6be59`](nodejs/node@f834a6be59)] - **deps**: update undici to 7.13.0 (Node.js GitHub Bot) [#59338](nodejs/node#59338) - \[[`db2417487e`](nodejs/node@db2417487e)] - **deps**: update sqlite to 3.50.4 (Node.js GitHub Bot) [#59337](nodejs/node#59337) - \[[`41978adb08`](nodejs/node@41978adb08)] - **deps**: V8: backport [`493cb53`](nodejs/node@493cb53691be) (Chengzhong Wu) [#59238](nodejs/node#59238) - \[[`05667991ca`](nodejs/node@05667991ca)] - **deps**: V8: backport [`1c3e018`](nodejs/node@1c3e018e7d48) (Renegade334) [#58818](nodejs/node#58818) - \[[`fd61588bb4`](nodejs/node@fd61588bb4)] - **doc**: rename x509.extKeyUsage to x509.keyUsage (Filip Skokan) [#59332](nodejs/node#59332) - \[[`a271ae4360`](nodejs/node@a271ae4360)] - **doc**: fix Pbkdf2Params hash attribute heading (Filip Skokan) [#59395](nodejs/node#59395) - \[[`72cfff165b`](nodejs/node@72cfff165b)] - **doc**: fix missing reference links for server.keepAliveTimeoutBuffer (Lee Jiho) [#59356](nodejs/node#59356) - \[[`8341916772`](nodejs/node@8341916772)] - **doc**: fix grammar in global dispatcher usage (Eng Zer Jun) [#59344](nodejs/node#59344) - \[[`e3e489706b`](nodejs/node@e3e489706b)] - **doc**: run license-builder (github-actions\[bot]) [#59343](nodejs/node#59343) - \[[`46527e8cea`](nodejs/node@46527e8cea)] - **doc**: correct orthography `eg.` → `e.g.` (Jacob Smith) [#59329](nodejs/node#59329) - \[[`d140c3713e`](nodejs/node@d140c3713e)] - **doc**: clarify the need of compiler compatible with c++20 (Rafael Gonzaga) [#59297](nodejs/node#59297) - \[[`95e9cabf9d`](nodejs/node@95e9cabf9d)] - **doc**: clarify release candidate stability index (Filip Skokan) [#59295](nodejs/node#59295) - \[[`a056dd36d2`](nodejs/node@a056dd36d2)] - **doc**: add WDYT to glossary (btea) [#59280](nodejs/node#59280) - \[[`1e2c52f5c4`](nodejs/node@1e2c52f5c4)] - **doc**: add manpage entry for --use-system-ca (Joyee Cheung) [#59273](nodejs/node#59273) - \[[`31a46fdeb4`](nodejs/node@31a46fdeb4)] - **doc**: add path.join and path.normalize clarification (Rafael Gonzaga) [#59262](nodejs/node#59262) - \[[`cff3725ff9`](nodejs/node@cff3725ff9)] - **doc**: fix typo in `test/common/README.md` (Yoo) [#59180](nodejs/node#59180) - \[[`31a9283591`](nodejs/node@31a9283591)] - **doc**: add note on process memoryUsage (fengmk2) [#59026](nodejs/node#59026) - \[[`5a98bff6b8`](nodejs/node@5a98bff6b8)] - **doc**: format safely for `doc-kit` (Aviv Keller) [#59229](nodejs/node#59229) - \[[`95b8b7ea5c`](nodejs/node@95b8b7ea5c)] - **domain**: remove deprecated API call (Alex Yang) [#59339](nodejs/node#59339) - \[[`2990f178bd`](nodejs/node@2990f178bd)] - **fs**: fix glob TypeError on restricted dirs (Sylphy-0xd3ac) [#58674](nodejs/node#58674) - \[[`e2fb4caf9c`](nodejs/node@e2fb4caf9c)] - **fs**: correct error message when FileHandle is transferred (Alex Yang) [#59156](nodejs/node#59156) - \[[`aeb4de55a7`](nodejs/node@aeb4de55a7)] - **(SEMVER-MINOR)** **fs**: port SonicBoom module to fs module as Utf8Stream (James M Snell) [#58897](nodejs/node#58897) - \[[`e79c93a5d0`](nodejs/node@e79c93a5d0)] - **(SEMVER-MINOR)** **http**: add server.keepAliveTimeoutBuffer option (Haram Jeong) [#59243](nodejs/node#59243) - \[[`0fb005a53f`](nodejs/node@0fb005a53f)] - **http2**: set Http2Stream#sentHeaders for raw headers (Darshan Sen) [#59244](nodejs/node#59244) - \[[`e055539604`](nodejs/node@e055539604)] - **lib**: add trace-sigint APIs (theanarkh) [#59040](nodejs/node#59040) - \[[`d2183d860a`](nodejs/node@d2183d860a)] - **lib**: optimize writable stream buffer clearing (Yoo) [#59406](nodejs/node#59406) - \[[`47543a7e17`](nodejs/node@47543a7e17)] - **lib**: handle windows reserved device names on UNC (Rafael Gonzaga) [#59286](nodejs/node#59286) - \[[`c6911f0717`](nodejs/node@c6911f0717)] - **lib**: do not modify prototype deprecated asyncResource (RafaelGSS) [#59195](nodejs/node#59195) - \[[`3c88b769bb`](nodejs/node@3c88b769bb)] - **lib**: restructure assert to become a class (Miguel Marcondes Filho) [#58253](nodejs/node#58253) - \[[`e91b54df59`](nodejs/node@e91b54df59)] - **lib**: handle superscript variants on windows device (Rafael Gonzaga) [#59261](nodejs/node#59261) - \[[`4ee467905d`](nodejs/node@4ee467905d)] - **lib**: use validateString (hotpineapple) [#59296](nodejs/node#59296) - \[[`c144d69efc`](nodejs/node@c144d69efc)] - **lib**: docs deprecate \_http\_\* (Sebastian Beltran) [#59293](nodejs/node#59293) - \[[`c89b67e681`](nodejs/node@c89b67e681)] - **lib**: add type names in source mapped stack traces (Chengzhong Wu) [#58976](nodejs/node#58976) - \[[`5b2363be8d`](nodejs/node@5b2363be8d)] - **lib**: prefer AsyncIteratorPrototype primordial (René) [#59097](nodejs/node#59097) - \[[`41b4f4d694`](nodejs/node@41b4f4d694)] - **meta**: clarify pr objection process further (James M Snell) [#59096](nodejs/node#59096) - \[[`0eb5962f1e`](nodejs/node@0eb5962f1e)] - **meta**: add mailmap entry for aditi-1400 (Aditi) [#59316](nodejs/node#59316) - \[[`a2b72c2304`](nodejs/node@a2b72c2304)] - **meta**: add tsc and build team as codeowners building.md (Rafael Gonzaga) [#59298](nodejs/node#59298) - \[[`d69f3ee1e0`](nodejs/node@d69f3ee1e0)] - **meta**: add nodejs/path to path files (Rafael Gonzaga) [#59289](nodejs/node#59289) - \[[`1e37eab865`](nodejs/node@1e37eab865)] - **node-api**: reword "implementation in an alternative VM" as implementable (Chengzhong Wu) [#59036](nodejs/node#59036) - \[[`64add6302a`](nodejs/node@64add6302a)] - **src**: use simdjson to parse SEA configuration (Joyee Cheung) [#59323](nodejs/node#59323) - \[[`e9c6636585`](nodejs/node@e9c6636585)] - **src**: mark realm leaf classes final (Anna Henningsen) [#59355](nodejs/node#59355) - \[[`42ef8147d1`](nodejs/node@42ef8147d1)] - **src**: warn about FastOneByteString invalidation (James M Snell) [#59275](nodejs/node#59275) - \[[`8686b8037a`](nodejs/node@8686b8037a)] - **src**: remove unused DSAKeyExportJob (Filip Skokan) [#59291](nodejs/node#59291) - \[[`1e5f632666`](nodejs/node@1e5f632666)] - **src**: use C++20 `contains()` method (iknoom) [#59304](nodejs/node#59304) - \[[`22d4683cfe`](nodejs/node@22d4683cfe)] - **src**: added CHECK\_NOT\_NULL check for multiple eq\_wrap\_async (F3lixTheCat) [#59267](nodejs/node#59267) - \[[`6a47ff4943`](nodejs/node@6a47ff4943)] - **src**: clear all linked module caches once instantiated (Chengzhong Wu) [#59117](nodejs/node#59117) - \[[`33728cb4ca`](nodejs/node@33728cb4ca)] - **src**: add nullptr checks in `StreamPipe::New` (Burkov Egor) [#57613](nodejs/node#57613) - \[[`4a907bdad1`](nodejs/node@4a907bdad1)] - **src**: add percentage support to --max-old-space-size (Asaf Federman) [#59082](nodejs/node#59082) - \[[`7c189d4f55`](nodejs/node@7c189d4f55)] - **test**: deflake sequential/test-tls-session-timeout (Joyee Cheung) [#59423](nodejs/node#59423) - \[[`fb0a6fb57f`](nodejs/node@fb0a6fb57f)] - **test**: exclude mock from coverage (Shima Ryuhei) [#59348](nodejs/node#59348) - \[[`7e10f95f13`](nodejs/node@7e10f95f13)] - **test**: split test-fs-cp.js (Joyee Cheung) [#59408](nodejs/node#59408) - \[[`41bcf5f659`](nodejs/node@41bcf5f659)] - **test**: update WPT resources,WebCryptoAPI,webstorage (Filip Skokan) [#59311](nodejs/node#59311) - \[[`f9f3dc94cb`](nodejs/node@f9f3dc94cb)] - **test**: add known issue test for fs.cpSync dereference bug (James M Snell) [#58941](nodejs/node#58941) - \[[`244d0c38a8`](nodejs/node@244d0c38a8)] - **test**: deflake stream-readable-to-web test (Ethan Arrowood) [#58948](nodejs/node#58948) - \[[`564e604a1a`](nodejs/node@564e604a1a)] - **test**: make test-inspector-network-resource sequential (Shima Ryuhei) [#59104](nodejs/node#59104) - \[[`7ab13b7477`](nodejs/node@7ab13b7477)] - **test**: don't use expose internals in test-http-outgoing-buffer.js (Meghan Denny) [#59219](nodejs/node#59219) - \[[`319df3859a`](nodejs/node@319df3859a)] - **test,crypto**: skip unsupported ciphers (Shelley Vohr) [#59388](nodejs/node#59388) - \[[`713c70c32a`](nodejs/node@713c70c32a)] - **test\_runner**: remove unused callee convertion (Alex Yang) [#59221](nodejs/node#59221) - \[[`e4ca30e115`](nodejs/node@e4ca30e115)] - **tools**: disable nullability-completeness warnings (Michaël Zasso) [#59392](nodejs/node#59392) - \[[`dab7f6b542`](nodejs/node@dab7f6b542)] - **tools**: check for std::vector\<v8::Local> in lint (Aditi) [#58497](nodejs/node#58497) - \[[`7b94982eb0`](nodejs/node@7b94982eb0)] - **tools**: allow selecting test subsystems with numbers in their names (Darshan Sen) [#59242](nodejs/node#59242) - \[[`16bbcd8881`](nodejs/node@16bbcd8881)] - **typings**: improve internal binding types (Nam Yooseong) [#59351](nodejs/node#59351) - \[[`76bc4d659b`](nodejs/node@76bc4d659b)] - **typings**: improve internal binding types (Michaël Zasso) [#59176](nodejs/node#59176) - \[[`eecd3272a6`](nodejs/node@eecd3272a6)] - **worker**: add name for worker (theanarkh) [#59213](nodejs/node#59213) - \[[`84c3513ce2`](nodejs/node@84c3513ce2)] - **worker**: implements nits in Web Locks code (Antoine du Hamel) [#59270](nodejs/node#59270) - \[[`bd68fbd753`](nodejs/node@bd68fbd753)] - **worker**: add cpuUsage for worker (theanarkh) [#59177](nodejs/node#59177) - \[[`201304537e`](nodejs/node@201304537e)] - **(SEMVER-MINOR)** **zlib**: add dictionary support to zstdCompress and zstdDecompress (lluisemper) [#59240](nodejs/node#59240) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels Aug 8, 2025

lib: optimize writable stream buffer clearing

8a8653d

Improved the `clearBuffer` function by replacing `buffered.splice` with `ArrayPrototypeSlice`. - Eliminates O(N) shifting overhead. - Improves CPU utilization and reduces GC overhead.

ri7116 force-pushed the fianlbuffer branch from 86dd5c8 to 8a8653d Compare August 8, 2025 13:18

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2025

mcollina approved these changes Aug 8, 2025

View reviewed changes

Ethan-Arrowood approved these changes Aug 8, 2025

View reviewed changes

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2025

gurgunday approved these changes Aug 8, 2025

View reviewed changes

gurgunday added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Aug 8, 2025

BridgeAR approved these changes Aug 8, 2025

View reviewed changes

addaleax approved these changes Aug 10, 2025

View reviewed changes

mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 11, 2025

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 11, 2025

nodejs-github-bot merged commit 8a2fec1 into nodejs:main Aug 11, 2025
64 checks passed

github-actions bot mentioned this pull request Aug 11, 2025

2025-08-12, Version 24.6.0 (Current) #59449

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

lib: optimize writable stream buffer clearing #59406

lib: optimize writable stream buffer clearing #59406

ri7116 commented Aug 8, 2025

Uh oh!

nodejs-github-bot commented Aug 8, 2025

Uh oh!

codecov bot commented Aug 8, 2025 •

edited

Loading

Uh oh!

mcollina commented Aug 8, 2025

Uh oh!

mcollina left a comment

Uh oh!

Ethan-Arrowood left a comment

Uh oh!

nodejs-github-bot commented Aug 8, 2025

Uh oh!

gurgunday left a comment

Uh oh!

Uh oh!

nodejs-github-bot commented Aug 11, 2025

Uh oh!

Uh oh!

Uh oh!

lib: optimize writable stream buffer clearing #59406

lib: optimize writable stream buffer clearing #59406

Conversation

ri7116 commented Aug 8, 2025

Uh oh!

nodejs-github-bot commented Aug 8, 2025

Uh oh!

codecov bot commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mcollina commented Aug 8, 2025

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

Ethan-Arrowood left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Aug 8, 2025

Uh oh!

gurgunday left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nodejs-github-bot commented Aug 11, 2025

Uh oh!

Uh oh!

codecov bot commented Aug 8, 2025 •

edited

Loading