[BUG] Inconsistency between calls to npm on node.js 16.x

[nassau-t]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

A call to npm outdated, if there are outdated packages, throws an exit code status of 1

C:\temp>node
Welcome to Node.js v16.11.0.
Type ".help" for more information.
> require('child_process').execSync('npm outdated -g --depth=0')
Uncaught Error: Command failed: npm outdated -g --depth=0
    at checkExecSyncError (node:child_process:826:11)
    at Object.execSync (node:child_process:900:15) {
  status: 1,
  signal: null,
  output: [
    null,
    <Buffer 50 61 63 6b 61 67 65 20 20 20 20 20 20 20 20 20 20 20 20 20 43 75 72 72 65 6e 74 20 20 20 57 61 6e 74 65 64 20 20 20 4c 61 74 65 73 74 20 20 4c 6f 63 ... 216 more bytes>,
    <Buffer >
  ],
  pid: 14996,
  stdout: <Buffer 50 61 63 6b 61 67 65 20 20 20 20 20 20 20 20 20 20 20 20 20 43 75 72 72 65 6e 74 20 20 20 57 61 6e 74 65 64 20 20 20 4c 61 74 65 73 74 20 20 4c 6f 63 ... 216 more bytes>,
  stderr: <Buffer >
}

C:\temp>npm -v
8.0.0

Expected Behavior

It was working like this in the the node.js 16.x branch until 16.11.

C:\temp>node
Welcome to Node.js v16.10.0.
Type ".help" for more information.
> require('child_process').execSync('npm outdated -g --depth=0')
<Buffer 50 61 63 6b 61 67 65 20 20 20 20 20 20 20 20 20 20 20 20 20 43 75 72 72 65 6e 74 20 20 20 57 61 6e 74 65 64 20 20 20 4c 61 74 65 73 74 20 20 4c 6f 63 ... 216 more bytes>

C:\temp>npm -v
7.24.0

Steps To Reproduce

1. In this environment...
   Windows 10 pro x64. Node.js v16.11.0 (with included npm v8.0.0)

2. With this config...
   A system with global outdated packages.

3. Run '...'
   require('child_process').execSync('npm outdated -g --depth=0')

4. See error...
   Throws a exit status code of 1

Environment

• OS: Windows 10 Pro x64
• Node: 16.11.0
• npm: 8.0.0

[nassau-t]

More details:

Call to npm outdated having outdated global packages

with node 16.11 and default npm 8.0.0

toni@MSI MINGW64 /c/temp
$ node -v
v16.11.0

toni@MSI MINGW64 /c/temp
$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
1
*Package             Current   Wanted   Latest  Location                         Depended by
@types/node         16.10.2  16.10.3  16.10.3  node_modules/@types/node         global
eslint               7.32.0    8.0.0    8.0.0  node_modules/eslint              global
webpack-dev-server    4.3.0    4.3.1    4.3.1  node_modules/webpack-dev-server  global*

with node 16.10 and default npm 7.24.0

toni@MSI MINGW64 ~
$ node -v
v16.10.0

toni@MSI MINGW64 ~
$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
0
*Package             Current   Wanted   Latest  Location                         Depended by
@types/node         16.10.2  16.10.3  16.10.3  node_modules/@types/node         global
eslint               7.32.0    8.0.0    8.0.0  node_modules/eslint              global
webpack-dev-server    4.3.0    4.3.1    4.3.1  node_modules/webpack-dev-server  global*

As you can see, exit status code has changed. When there are outdated packages, npm 7.24.0 exit with an exit status of 0, and npm 8.0.0 exit with an exit status of 1.

Call to npm outdated without having outdated global packages

with node 16.11 and default npm 8.0.0

toni@MSI MINGW64 ~
$ node -v
v16.10.0

toni@MSI MINGW64 ~
$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
0
**

with node 16.10 and default npm 7.24.0

toni@MSI MINGW64 ~
$ node -v
v16.11.0

toni@MSI MINGW64 ~
$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
0
**

So, when there are no outdated packages, both exit with status code 0.

Summary

I understand that exit error code must be 0 if it's ok, or 1-255 if there are an error.
When I call npm outdated and there are outdated packages, there is no failure or error. Why it return a 1 exit code?

The usual way to do that should be to exit with an error code of 0 (if there are no error), and then, if you want to know if there are outdated packages or not, read the return string (as showed on the examples), and if the string is a void string, there are no outdated packages.

[ljharb]

This changed in npm v7.24.2 - npm v7.24.1 and below exit zero in this case.

[nassau-t]

I'm astonished.
How it's possible that anyone can return an error (exit codes 1-255) when there aren't any error.
In my thought it is a lot more consistent to return an ok (exit code 0) and if someone want to know if there are no results, it's very easy, because the return string would be an empty string.

without outdated packages

$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
0
**

with outdated packages

$ result=$(npm outdated -g --depth=0); echo $?; echo "*$result*"
0
*Package             Current   Wanted   Latest  Location                         Depended by
@types/node         16.10.2  16.10.3  16.10.3  node_modules/@types/node         global
eslint               7.32.0    8.0.0    8.0.0  node_modules/eslint              global
webpack-dev-server    4.3.0    4.3.1    4.3.1  node_modules/webpack-dev-server  global*

[ljharb]

@nassau-t that depends on what mental model you have of this command. If it's "get me the results of an outdated check", then it should return 0, because it got the results. If it's "tell me if everything is up to date", then it absolutely should return nonzero when there's nonzero outdated deps.

[nassau-t]

I will prove it empirically:

C:\temp>node
Welcome to Node.js v16.11.0.
Type ".help" for more information.
> require('child_process').execSync('npm outdated -g --depth=0')
Uncaught Error: Command failed: npm outdated -g --depth=0
    at checkExecSyncError (node:child_process:826:11)
    at Object.execSync (node:child_process:900:15) {
  status: 1,
  signal: null,
  output: [
    null,
    <Buffer 50 61 63 6b 61 67 65 20 20 20 20 20 20 20 20 20 20 20 20 20 43 75 72 72 65 6e 74 20 20 20 57 61 6e 74 65 64 20 20 20 4c 61 74 65 73 74 20 20 4c 6f 63 ... 216 more bytes>,
    <Buffer >
  ],
  pid: 14996,
  stdout: <Buffer 50 61 63 6b 61 67 65 20 20 20 20 20 20 20 20 20 20 20 20 20 43 75 72 72 65 6e 74 20 20 20 57 61 6e 74 65 64 20 20 20 4c 61 74 65 73 74 20 20 4c 6f 63 ... 216 more bytes>,
  stderr: <Buffer >
}

C:\temp>npm -v
8.0.0

What do you see here? Because I see an error.

In other words, exit status code has nothing to do with what you think or what you expect. It only makes reference to if the execution is ok or there is an error in the execution, nothing more.

[ljharb]

Right - but “the execution” is subjective.

[nassau-t]

No, it's not subjective.

It's not subjective because an exit status code of 0 is execution were ok, and an exit status code of 1-255 is there were an error on the execution. Because of that, the system says there is an error. It's not me, and neither are you. It's totally objective because that is the rule.

Look at the example above, I have proved it.

[ljharb]

There is no rule, it’s merely a convention, and every tool is always free to choose as they see fit.

An analogue might be, when i do rm nonexistentPath, it’s actually very annoying that rm errors there. I’m not actually asking it to succeed when that path becomes unlinked, I’m asking it to make sure that path is unlinked - i don’t actually care if the path existed before or not.

[nassau-t]

I will repeat the empirical proof again.

C:\temp>node
Welcome to Node.js v16.11.0.
Type ".help" for more information.
> try {
...   let result = require('child_process').execSync('npm outdated -g --depth=0');
...   console.log('ok');
... } catch (err) {
...   console.log('error');
... }
error
undefined

What do you see here? Because I see an error.
And in every language (java, python...) that you execute that, it will end with an error. Simply because npm outdated has exit with error code 1, that is an error.

I repeat: exit status code has nothing to do with what you think or what you expect. It only makes reference to if the execution is ok or there is an error in the execution, nothing more.

[ljharb]

And in one view, when there are outdated packages, there’s an error in execution, and one absolutely and only wants the thrown error in your case.

I’m not actually arguing in favor of the exit code - I’m objecting to the incorrect assertion that there’s an obviously correct answer here.