npm: last nightly / v8-canary make `npm update` unusable

[vsemozhetbyt]

• Version: Node.js master (10.0.0 after 2018-01-15, see npm: last nightly / v8-canary make npm update unusable #19405 (comment)), npm 5.6.0
• Platform: Windows 7 x64 (maybe all)
• Subsystem: npm

Please, check if this is reproducible:

1. Install/download last nightly or v8-canary build.

2. Run npm update or npm update -g for a package with many dependencies (see an example in npm: last nightly / v8-canary make npm update unusable #19405 (comment)).

I have 1.5 GB memory consumption, ~100% CPU, and never-ending process without any output messages.

Installing / using Node.js v9.8.0 with the same npm fixes the issue.

[vsemozhetbyt]

The issue seems to appear if something does need to be updated: after successfully updating (npm update -g) with Node.js v9.8.0, npm update -g returns immediately with Node.js v10.0.0, but npm update with outdated packages hangs.

[vsemozhetbyt]

Narrowed case.

https://github.com/eslint/eslint/releases — last versions: 4.18.2, 4.19.0.

md test-npm && cd test-npm
npm install eslint@4.18.2
npm update

The last command hangs trying to update to 4.19.0.

[targos]

can you run it in verbose mode so we have a better idea about when it starts hanging?

[vsemozhetbyt]

j:\temp\test-npm>npm update --verbose
npm info it worked if it ends with ok
npm verb cli [ 'C:\\Program Files\\nodejs\\node.exe',
npm verb cli   'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
npm verb cli   'update',
npm verb cli   '--verbose' ]
npm info using npm@5.6.0
npm info using node@v10.0.0-v8-canary201803169dc1b12f4d
npm verb npm-session 5aebf4c4722b2950
npm verb update computing outdated modules to update
npm verb request uri https://registry.npmjs.org/eslint
npm verb request no auth needed
npm info attempt registry request try #1 at 16:17:48
npm verb request using bearer token for auth
npm verb request id f1120122477c2bf9
npm http request GET https://registry.npmjs.org/eslint
npm http 200 https://registry.npmjs.org/eslint
npm verb headers { 'content-type': 'application/json; charset=UTF-8',
npm verb headers   server: 'UploadServer',
npm verb headers   'cache-control': 'max-age=300',
npm verb headers   'last-modified': 'Sat, 17 Mar 2018 14:17:19 GMT',
npm verb headers   etag: '"5aad236f-8c747"',
npm verb headers   'x-npm-region': 'EU-East',
npm verb headers   'content-encoding': 'gzip',
npm verb headers   'content-length': '35168',
npm verb headers   'accept-ranges': 'bytes',
npm verb headers   date: 'Sat, 17 Mar 2018 14:17:50 GMT',
npm verb headers   via: '1.1 varnish',
npm verb headers   age: '21',
npm verb headers   connection: 'keep-alive',
npm verb headers   'x-served-by': 'cache-hhn1528-HHN',
npm verb headers   'x-cache': 'HIT',
npm verb headers   'x-cache-hits': '1',
npm verb headers   'x-timer': 'S1521296270.344240,VS0,VE1',
npm verb headers   vary: 'Accept-Encoding, Accept' }

Hangs here.

[vsemozhetbyt]

Some debugging data:

v10.0.0-nightly2018011585739b6c5b is OK
v10.0.0-nightly20180116f75bc2c1a5 hangs

So the cause seems to be here: 85739b6...f75bc2c

These frames are the beginning of the infinite spin:

node/deps/npm/lib/update.js

Line 53 in 1329844

log.info('outdated', 'updating', wanted)

->

node/deps/npm/node_modules/npmlog/log.js

Line 286 in 1329844

return this.log.apply(this, a)

->

node/deps/npm/node_modules/npmlog/log.js

Line 194 in 1329844

message = util.format.apply(util, a)

From there, the data in recursive chains of util.format()/util.inspect() and their sub-calls begins to differ.

So maybe #17907 is the cause.

We may need to fix this before v10 release.

cc @nodejs/util

[BridgeAR]

This seems to be a formerly hidden bug in npm and not a bug in Node.js.

I do understand that this is something we have to find a solution for soon though. We might just use a high depth limit to circumvent issue like these but that is more like a hack than anything else and could only be a intermediate step. It will still be a recursive call up to the maximum limit and anyone who changes the default to unlimited is also going to run into this.

We could set the limit to e.g., 1000 for now to circumvent the issue. I am just not a fan of doing so because it will prevent anyone else from realizing that there is an issue with their code if they also have a unlimited recursion.

@vsemozhetbyt are you fine with opening a PR against npm to fix the issue? You already know how this works, so I guess it should be relatively straight forward?