npmlock2nix.node_modules started failing to populate artifacts

[flokli]

A bump of nixpkgs, most likely the bump of nodejs from 14.x to 16.x (4c60ee3da1e) started causing npmlock2nix.node_modules to fail to populate the sources specified in package-lock.json.

https://nodesource.com/blog/whats-new-in-npm-8 mentions there's a new lockfile format, and npm seems to do a "one-time fixup" of the old lockfile.

This fails in the sandbox, as it obviously can't write back to that file.

If I invoke a npm install outside of the sandbox, it mentions fetching more metadata:

npm WARN old lockfile 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 

With the updated npm-lock.json, I then run the build again, and it fails to discover the provided tarballs:

       last 10 log lines:
       > patching sources
       > configuring
       > no configure script, doing nothing
       > building
       > npm ERR! code ENOTCACHEDmsill reify mark deleted [no
       > npm ERR! request to https://registry.npmjs.org/zwitch/-/zwitch-1.0.5.tgz failed: cache mode is 'only-if-cached' but no cached response is available.
       > 
       > npm ERR! A complete log of this run can be found in:
       > npm ERR!     /build/.npm/_logs/2022-01-25T10_17_37_004Z-debug.log
       > 

As of now, I was able to workaround this by passing nodejs = nodejs-14_x; in the call to npmlock2nix.node_modules.

[happysalada]

Experiencing the same.
I've tried adding the missing packages directly to the package.json but to no avail.
Workaround is great though, thank you!

[zimbatm]

Apparently the dependencies now appear twice in the package-lock.json, and npm is reading the former one who isn't getting patched:

{
  "name": "my-project",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "my-project",
      "dependencies": {
        "@ampproject/remapping": "^2.0.3",
      },
      "devDependencies": { }
    },
    "node_modules/@ampproject/remapping": {
      "version": "2.0.3",
      "resolved": "https://registry.npmjs.org/@ampproject/remapping/-/remapping-2.0.3.tgz",
      "integrity": "sha512-DmIAguV77yFP0MGVFWknCMgSLAtsLR3VlRTteR6xgMpIfYtwaZuMvjGv5YlpiqN7S/5q87DHyuIx8oa15kiyag==",
      "peer": true,
      "dependencies": {
        "@jridgewell/sourcemap-codec": "^1.4.9",
        "@jridgewell/trace-mapping": "^0.2.7"
      },
      "engines": {
        "node": ">=6.0.0"
      }
    }
  },
  "dependencies": {
    "@ampproject/remapping": {
      "version": "2.0.3",
      "resolved": "https://registry.npmjs.org/@ampproject/remapping/-/remapping-2.0.3.tgz",
      "integrity": "sha512-DmIAguV77yFP0MGVFWknCMgSLAtsLR3VlRTteR6xgMpIfYtwaZuMvjGv5YlpiqN7S/5q87DHyuIx8oa15kiyag==",
      "peer": true,
      "requires": {
        "@jridgewell/sourcemap-codec": "^1.4.9",
        "@jridgewell/trace-mapping": "^0.2.7"
      }
    }
  }
}

[poelzi]

So, whats the plan. I can't use node 14 and node 16 is broken. Should I ditch npmlock2nix and try the next thing ?

[andir]

"the plan" is to find time/motivation/someone to dive into the issue with npm16. We need to find the differences in the lock files and figure out a way to do our shebang patching just like before.

[jerith666]

I think the fact that your dependencies are marked as peer: true is important. https://docs.npmjs.com/cli/v7/configuring-npm/package-json#peerdependencies says:

In npm versions 3 through 6, peerDependencies were not automatically installed, and would raise a warning if an invalid version of the peer dependency was found in the tree. As of npm v7, peerDependencies are installed by default.

[flokli]

Can this be closed, sor the same reasons as #153 (comment)?