-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
test: deflake stream-readable-to-web test #58948
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
import { mustCall } from '../common/index.mjs'; | ||
import { Readable } from 'node:stream'; | ||
import { memoryUsage } from 'node:process'; | ||
import assert from 'node:assert'; | ||
import { setImmediate } from 'node:timers/promises'; | ||
|
||
// Based on: https://github.com/nodejs/node/issues/46347#issuecomment-1413886707 | ||
// edit: make it cross-platform as /dev/urandom is not available on Windows | ||
|
||
const MAX_MEM = 256 * 1024 * 1024; // 256 MiB | ||
|
||
function checkMemoryUsage() { | ||
assert(memoryUsage().arrayBuffers < MAX_MEM); | ||
} | ||
|
||
const MAX_BUFFERS = 1000; | ||
let buffersCreated = 0; | ||
|
||
const randomNodeStream = new Readable({ | ||
read(size) { | ||
if (buffersCreated >= MAX_BUFFERS) { | ||
this.push(null); | ||
return; | ||
} | ||
|
||
this.push(Buffer.alloc(size)); | ||
buffersCreated++; | ||
} | ||
}); | ||
|
||
randomNodeStream.on('error', (err) => { | ||
assert.fail(err); | ||
}); | ||
|
||
// Before doing anything, make sure memory usage is okay | ||
checkMemoryUsage(); | ||
|
||
// Create stream and check memory usage remains okay | ||
|
||
const randomWebStream = Readable.toWeb(randomNodeStream); | ||
|
||
checkMemoryUsage(); | ||
|
||
let timeout; | ||
try { | ||
// Wait two seconds before consuming the stream to see if memory usage increases | ||
timeout = setTimeout(mustCall(async () => { | ||
// Did the stream leak memory? | ||
checkMemoryUsage(); | ||
// eslint-disable-next-line no-unused-vars | ||
for await (const _ of randomWebStream) { | ||
// Yield event loop to allow garbage collection | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FWIW, this behavior likely relies on the bug described in #54918 - the event loop is not supposed to block on just any garbage collection, which can cause a deadlock, and in the future it might go away (if someone manages to eradicate the deadlock by rewriting the task runner). Alternatively if there's a specific leaky object with a known class that this tries to test, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good to know. I agree that issue seems related here; particularly with the previous heavy use of timers. I believe that the Readable / GC is doing what they are supposed to regarding memory management. |
||
await setImmediate(); | ||
// consume the stream | ||
// check memory usage remains okay | ||
checkMemoryUsage(); | ||
} | ||
}), 2000); | ||
} catch (err) { | ||
if (timeout) { | ||
clearTimeout(timeout); | ||
} | ||
assert.fail(err); | ||
} |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We start consuming immediately now, this is a significant change. See #46347 (comment).