[BUG] unable to resolve dependency tree

[edwinm]

Current Behavior:

When I run npm i @typescript-eslint/parser@latest, I get:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found: @typescript-eslint/parser@3.10.1
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   dev @typescript-eslint/parser@"4.5.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! dev @typescript-eslint/parser@"4.5.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/edwinm/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/edwinm/.npm/_logs/2020-10-20T20_34_49_853Z-debug.log

Expected Behavior:

I expect npm to upgrade my package. Npm says "Fix the upstream dependency conflict", but I don't see any.

If npm sees a conflict, it should explain where the conflict is.

Steps To Reproduce:

git clone https://github.com/edwinm/carbonium.git
cd carbonium
npm update
npm i @typescript-eslint/parser@latest

Environment:

MacOS 10.15.7
Node v15.0.0
npm 7.0.2

[isaacs]

Huh, yeah, that isn't a very helpful explanation, is it?

Can you share your package.json file?

[isaacs]

Oh, sorry, you posted repro steps, sorry I'd missed that. Taking a look now.

[isaacs]

Hmm... this seems to work for me using npm 7.0.2 and 7.0.3 (published today).

Can you share the full debug log at /Users/edwinm/.npm/_logs/2020-10-20T20_34_49_853Z-debug.log and the eresolve report at /Users/edwinm/.npm/eresolve-report.txt? That might shed some light on it.

Also, what git commit are you at in carbonium? Is it possible this has been fixed?

$ git clone https://github.com/edwinm/carbonium.git
Cloning into 'carbonium'...
remote: Enumerating objects: 13037, done.
remote: Counting objects: 100% (13037/13037), done.
remote: Compressing objects: 100% (8729/8729), done.
remote: Total 13037 (delta 3499), reused 12934 (delta 3403), pack-reused 0
Receiving objects: 100% (13037/13037), 23.91 MiB | 15.89 MiB/s, done.
Resolving deltas: 100% (3499/3499), done.

$ cd carbonium
/Users/isaacs/dev/npm/arborist/x/carbonium

$ npm update
npm i @typescript-eslint/parser@latest

added 569 packages, and audited 569 packages in 13s

59 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

$ npm i @typescript-eslint/parser@latest
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!

> husky@4.3.0 preuninstall /Users/isaacs/dev/npm/arborist/x/carbonium/node_modules/husky
> node husky uninstall

husky > Uninstalling git hooks
husky > Done
npm WARN @typescript-eslint/parser@4.5.0 requires a peer of eslint@^5.0.0 || ^6.0.0 || ^7.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN tsutils@3.17.1 requires a peer of typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta but none is installed. You must install peer dependencies yourself.

+ @typescript-eslint/parser@4.5.0
added 12 packages from 19 contributors, removed 546 packages, updated 23 packages and audited 35 packages in 6.273s

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

[isaacs]

Oh, whoops that last install was in npm v6. trying again, I get this:

$ npm i @typescript-eslint/parser@latest
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found: @typescript-eslint/parser@3.10.1
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   dev @typescript-eslint/parser@"4.5.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! dev @typescript-eslint/parser@"4.5.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/isaacs/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/isaacs/.npm/_logs/2020-10-20T21_42_51_028Z-debug.log

This is weird, it's not a peer conflict, looks like, it's just refusing to clobber an existing dev dep with a new version. That's a bug, yes.

[isaacs]

Aha, no, it is a peer conflict, but it's failing to properly report on it, because we ignore other edgesIn when the dep node is a dependency of the root node. https://github.com/npm/arborist/blob/main/lib/node.js#L364-L375

I recall that we added that because it got quite noisy in some cases, but it seems clear to me that was a mistake. When that check is removed, I see a report like this, which is much clearer:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found: @typescript-eslint/parser@3.10.1
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   peer @typescript-eslint/parser@"^3.0.0" from @typescript-eslint/eslint-plugin@3.10.1
npm ERR!   node_modules/@typescript-eslint/eslint-plugin
npm ERR!     dev @typescript-eslint/eslint-plugin@"^3.10.1" from the root project
npm ERR!   dev @typescript-eslint/parser@"4.5.0" from the root project

If you do npm i @typescript-eslint/parser@latest @typescript-eslint/eslint-plugin@latest, updating both dependencies, I would have expected it to work, but in that case, it runs into an issue replacing one while the other is present, even though it's invalid and about to be replaced! My minimized testcase doesn't fail on this, I believe because eslint-plugin comes alphabetically ahead of parser. Building a new simplified test case now.

[isaacs]

Ahahhhh... this doesn't have anything to do with the order of the deps. The (second) issue is that both packages have a peer dependency on estlint, and the root package has an eslint dep as well.

So, first issue, the report isn't helpful. That's addressed by leaving in the other dependencies when a dep node has an edge coming in from the root package. Easy enough.

That would make it clear that you can't simply do npm i @typescript-eslint/parser@latest, because the @typescript-eslint/eslint-plugin package also has to be updated along with it.

Fair enough, so you do npm i @typescript-eslint/parser@latest @typescript-eslint/eslint-plugin@latest to update them both.

Currently that crashes at the phase where it tries to determine of the intended target location has any current dependencies which will collide with one of the peers of the dependency you're trying to add, leading to this loop here.

This finds eslint in the peerSet of the module about to be added, and checks to see if there is an eslint there already. Is that eslint currently valid? Ok, I guess we can't replace it then!

Except... the one we're trying to add also has an eslint of the exact same version.

This fixes the issue:

diff --git a/lib/arborist/build-ideal-tree.js b/lib/arborist/build-ideal-tree.js
index ae9965e..d85d858 100644
--- a/lib/arborist/build-ideal-tree.js
+++ b/lib/arborist/build-ideal-tree.js
@@ -1351,6 +1351,11 @@ This is a one-time fix-up, please be patient...
             // only respect valid edges, however, since we're likely trying
             // to fix the very one that's currently broken!
             if (edge.from === target && edge.valid) {
+              // if we have a replacement in the peerSet that we're trying
+              // to place, and the replacement will be fine, then that's ok.
+              const replacement = dep.parent.children.get(edge.name)
+              if (replacement && replacement.satisfies(edge))
+                continue
               canReplace = false
               break OUTER
             }

Will have a PR up as soon as I have a test for this.

Minimized repro case:

{
  "dependencies": {
    "@isaacs/testing-peer-dep-conflict-chain-a": "1",
    "@isaacs/testing-peer-dep-conflict-chain-single-a": "^1.0.1",
    "@isaacs/testing-peer-dep-conflict-chain-single-b": "^1.0.1"
  }
}

npm install
npm install @isaacs/testing-peer-dep-conflict-chain-single-a@2 @isaacs/testing-peer-dep-conflict-chain-single-b@2