Skip to content

test: fix timeout in sequential/test-fs-watch-system-limit #23560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 180 commits into from

Conversation

mmmscott
Copy link
Contributor

During run of make test on Linux, it is possible for this test
to spawn 200 processes and hang since there is a check to max
out at 200 child processes.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

During run of make test on Linux, it is possible for this test
to spawn 200 processes and hang since there is a check to max
out at 200 child processes.
@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label Oct 12, 2018
@targos targos added the code-and-learn Issues related to the Code-and-Learn events and PRs submitted during the events. label Oct 12, 2018
@BridgeAR BridgeAR changed the title test: fix timeout when running sequential/test-fs-watch-system-limit.js test: fix timeout in sequential/test-fs-watch-system-limit Oct 12, 2018
addaleax and others added 24 commits October 12, 2018 21:07
Original commit message:

    [api] Remove deprecated wasm methods

    These methods were deprecated in 7.0, now we can remove them.

    [email protected]

    Bug: v8:7868
    Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
    Change-Id: I60badb378a055152bdd27aed67d11ddf74fce174
    Reviewed-on: https://chromium-review.googlesource.com/1209283
    Reviewed-by: Adam Klein <[email protected]>
    Commit-Queue: Clemens Hammacher <[email protected]>
    Cr-Commit-Position: refs/heads/master@{nodejs#55695}

Refs: v8/v8@b0af309

PR-URL: nodejs#23415
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: nodejs#23455
Reviewed-By: Tobias Nießen <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Covering the case when fs-read get invalid argument for file handle

PR-URL: nodejs#23601
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Minor cleanup in the lifetime for the platform worker initialization
synchronization barrier.

PR-URL: nodejs#23419
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
When using `assert.strictEqual`, the first argument must be the actual
value and the second argument must be the expected value.

PR-URL: nodejs#23448
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23449
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Have actual first, expected second.

PR-URL: nodejs#23450
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Tobias Nießen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reverse the argument for assertion. The first argument should be the
actual value and the second value should be the expected value. When
there is an AssertionError, the expected and actual value will be
labeled correctly.

PR-URL: nodejs#23451
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Catch statement defines err variable that is never used, so it
is safe to remove that.

PR-URL: nodejs#23452
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Converts RangeError assertions to use common.expectsError and includes
an assertion for the error code.

PR-URL: nodejs#23454
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23456
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23457
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23458
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Hitesh Kanwathirtha <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23459
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23461
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23463
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23465
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
test-http-expect-handling.js has an instance of the test value and the
expected value being reversed.

Refs: https://nodejs.org/api/assert.html

PR-URL: nodejs#23466
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23468
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
On success case, timeout still tries to fire and write to
stream. Only write to stream if we haven't finished
During run of make test on Linux, it is possible for this test
to spawn 200 processes and hang since there is a check to max
out at 200 child processes.
On success case, timeout still tries to fire and write to
stream. Only write to stream if we haven't finished
binaryme and others added 15 commits October 15, 2018 15:34
fixed order of parameters in assert.strictEqual() assertion functions,
first argument provided was the expected value and the second value
was the actual value.

this is backwards from the documentation for assertions like
assert.strictEqual() where the first value being tested and the second
value is the expected value

PR-URL: nodejs#23609
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23608
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23607
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23604
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Change base64_encoded_size and unbase64 to inline functions. The
base64_encoded_size is a constexpr to be used in function declarations.

PR-URL: nodejs#23603
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23602
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Fix assert.strictEquals argument order.

Arguments were actual first, expected second, contrary to
the documentation. Now, actual value is first and expected
value is second.

PR-URL: nodejs#23600
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23599
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23598
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
PR-URL: nodejs#23594
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Added test coverage for 4 un-covered if statements in slowCases

PR-URL: nodejs#23592
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
On my machine, this brings test execution time down from about 2
seconds to 0.2 seconds.

PR-URL: nodejs#23430
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Tobias Nießen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
While it doesn't make any difference now. In the future PromiseHooks
could be refactored to provide an asyncId instead of the promise object.
That would make escape analysis on promises possible.

Escape analysis on promises could lead to a more efficient destroy hook,
if provide by PromiseHooks as well. But at the very least would allow
the destroy hook to be emitted earlier. The destroy hook not being
emitted on promises frequent enough is a known and reported issue.
See nodejs#14446 and
Jeff-Lewis/cls-hooked#11.

While all this is speculation for now, it all depends on the promise
object not being a part of the PromiseWrap resource object.

Ref: nodejs#14446
Ref: nodejs/diagnostics#188

PR-URL: nodejs#23443
Refs: nodejs#14446
Refs: nodejs/diagnostics#188
Reviewed-By: Benedikt Meurer <[email protected]>
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: George Adams <[email protected]>
Add test to check:
- for `null` as len parameter
- if error is propagated into callback if file doesn't exist
- if an error is actually thrown if len is not a number

PR-URL: nodejs#23620
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Split out things that are specific to zlib as a specific
compression library, vs. the interface that is common to
most C compression libraries.

This should pave the way for including support for e.g.
brotli.

PR-URL: nodejs#23360
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Daniel Bevenius <[email protected]>
const timeoutStream = new stream.PassThrough();
timeoutStream.write('Error: timeout');
timeoutStream.end();
timeoutStream.pipe(gatherStderr);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not call gatherStderr.write() directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call!

timeoutStream.pipe(gatherStderr);

finished = true;
processes.forEach((proc) => proc.kill());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? Shouldn’t it already be happening below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit aggressive with the process killing since the test had left my machine with a lot of child processes, but you're right

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be good to add to process.once('exit') in case the test fails in the middle.

let accumulated = '';
gatherStderr.on('data', common.mustCallAtLeast((chunk) => {
accumulated += chunk;
if (accumulated.includes('Error:') && !finished) {
assert(
accumulated.includes('ENOSPC: System limit for number ' +
'of file watchers reached') ||
accumulated.includes('EMFILE: '),
accumulated.includes('EMFILE: ') ||
accumulated.includes('Timeout'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spells Timeout with a capital T, the text above has a lowercase t, so they don’t match … is that intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unintentional, thank you

dtoki and others added 10 commits October 16, 2018 13:46
PR-URL: nodejs#23597
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Сковорода Никита Андреевич <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Fix strictEqual assertion arguments order to conform
to the function signature in buffer tests

PR-URL: nodejs#23486
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
During run of make test on Linux, it is possible for this test
to spawn 200 processes and hang since there is a check to max
out at 200 child processes.
On success case, timeout still tries to fire and write to
stream. Only write to stream if we haven't finished
During run of make test on Linux, it is possible for this test
to spawn 200 processes and hang since there is a check to max
out at 200 child processes.
On success case, timeout still tries to fire and write to
stream. Only write to stream if we haven't finished
On success case, timeout still tries to fire and write to
stream. Only write to stream if we haven't finished
@mmmscott
Copy link
Contributor Author

sorry, i messed up this rebase completely. i'm going to close this PR and redo it.

@mmmscott mmmscott closed this Oct 16, 2018
@mmmscott mmmscott deleted the fix-fs-watch-limit-timeout branch October 16, 2018 14:10
@refack
Copy link
Contributor

refack commented Oct 16, 2018

@marcusscott in case you still have trouble rebasing, you could reopen this PR, and I could help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-and-learn Issues related to the Code-and-Learn events and PRs submitted during the events. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.