libuv update in 18.18.0 breaks webpack's `thread-loader`

[anomiex]

Version

v18.18.0

Platform

Linux abulia 6.5.0-1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.3-1 (2023-09-13) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Starting in an empty directory,

1. Create the following files:
   • package.json:

     {
     	"dependencies": {
     		"thread-loader": "^4.0.2",
     		"webpack": "^5.88.2",
     		"webpack-cli": "^4.10.0"
     	}
     }

   • webpack.config.js:

     const path = require( 'path' );
     
     module.exports = {
         mode: 'production',
         entry: './index.js',
         output: {
             path: path.resolve( __dirname, 'dist' ),
             filename: 'foo.bundle.js',
         },
         module: {
             rules: [
                 {
                     test: /\.(js)$/,
                     exclude: /node_modules/,
                     use: [
                         {
                             loader: require.resolve( 'thread-loader' ),
                         },
                     ],
                 }
             ]
         },
     };
     

2. Run node -e 'console.log( "console.log( typeof {} );" ); for(let i=0; i<5000; i++){ console.log( "// we need a large file in order to trigger the bug" ); }' > index.js. Increase the "5000" if necessary, I don't know whether it depends on the system in some manner.
3. npm install
4. npm exec webpack

How often does it reproduce? Is there a required condition?

Always. No requirements I know of beyond what's described above and being on Linux.

What is the expected behavior? Why is that the expected behavior?

asset foo.bundle.js 22 bytes [emitted] [minimized] (name: main)
./index.js 254 KiB [built] [code generated]
webpack 5.88.2 compiled successfully in 376 ms

Because that's what happens when webpack runs successfully.

What do you see instead?

Process hangs before producing any output. CPU is not being used, processes seem to be waiting in epoll_pwait.

Additional information

It worked in 18.17.1. fb2b80f appears to be the first failing commit.

Setting UV_USE_IO_URING=0 works around the problem. Of course, that's undocumented and temporary so a real fix would be good.

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

So my guess is that the new io_uring stuff started returning short writes when passed a lot of data, which WriteStream didn't handle correctly until 88ba79b fixed it.

See also webpack-contrib/thread-loader#191

[bnoordhuis]

The bug fix will be backported to v18.x automatically, it's just a matter of waiting until it shows up in a release.

[matthewp]

In Astro we had to pin to 18.17.1 after GitHub upgraded its CI to use 18.18. Not getting good stacktraces or failures, just seems to hang. Sounds similar to this issue.

[richardlau]

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

I've cherry-picked 88ba79b (and a4928b0 which it depends on to land cleanly) onto v18.x-staging so it's ready for the next non-security Node.js 18 release. Tentatively we have a release planned for October, see nodejs/Release#737 for the plan, but bear in mind the plan is subject to change based on releaser availability.

FYI @nodejs/lts @nodejs/releasers

[dnalborczyk]

I'm wondering, since this appears to be a known bug (for a while?) in 20.x, and v18 is currently the only LTS line, shouldn't have the offending commit be either omitted from the v18.18.0 release, or the v18.18.0 release be post-poned while waiting for the fix, or a bugfix release being made now with v18.18.1, as opposed to be waiting for a 18.19.0 release sometime in October?

It's just a bit surprising, but I also don't have the entire context.