Module Hooks cannot be registered from worker thread in 22.2.0+

[ShogunPanda]

Version

v22.2.0

Platform

Darwin panda.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:49 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6020 arm64

Subsystem

module

What steps will reproduce the bug?

Create the following files:

index.js

const { Worker } = require("worker_threads");

new Worker("./lib/child.js", {
  execArgv: ["--import", "./lib/register.js"],
});

lib/register.js

const { register } = require("node:module");
const { pathToFileURL } = require("node:url");

register("./hooks.mjs", pathToFileURL(__filename));

Then execute: node index.js

How often does it reproduce? Is there a required condition?

Always.

What is the expected behavior? Why is that the expected behavior?

The worker thread should start with the hooks register successfully.

What do you see instead?

The process hangs.
Removing the register call will solve the problem.

Additional information

This problem happens due to #52706, as already triaged in #53097.

An additional problem is that the suggested workaround forces people to use node --import register.js in order to solve the problem as the module thread is shared with Workers.
While this is fine, there might be situation in which users cannot use --import (or --require).
In those situation I have observed the register call will successfully work for subsequent dynamic import() call but NOT for require() (unless I missed something obvious).

I created https://github.com/ShogunPanda/hooks-repro to reproduce this.

[ShogunPanda]

CC: @mcollina @dygabo @GeoffreyBooth

[dygabo]

Hi @ShogunPanda,
the example tries to call module.register from the worker thread, which is currently not supported as pointed out by you and as mentioned in the other issue. Why not calling the module.register from the main thread and the hooks defined in hooks.js would be active also for any worker threads that will be spawned. From my perspective that was the intended behaviour of #52706. For supporting module.register from a worker thread we decided in that discussion to postpone to a followup and the documentation PR is meant just as a temporary clarification until that is solved.

index.js:

const { register } = require("node:module");
const { pathToFileURL } = require("node:url");
const { Worker } = require("worker_threads");

register("./hooks.mjs", pathToFileURL(__filename));

new Worker("./lib/child.js");

and then node index.js should work, the hooks in hooks.js are imported and they are active also on the instantiated Worker.

Would that work in your case?

[targos]

The title of this issue seems wrong, if it's about registering from a worker thread.

[ShogunPanda]

@targos It was a mental typo. Fixed. Thanks.

[mcollina]

@dygabo none of the limitations introduced by #52706 were listed in the OP.

[ShogunPanda]

@dygabo I can see that, but the problem is that the process hangs rather than immediately throwing an exception.

[GeoffreyBooth]

@nodejs/loaders

[GeoffreyBooth]

So there are two solutions that I can think of:

1. Calling register from new Worker execArgv --import throws an error, and we document that this is expected.

2. Calling register from new Worker execArgv --import or from a worker thread behaves the same as it would if called from the main thread: a hooks thread is created if it doesn’t already exist, and the new hooks are added. The hooks apply to all threads.

I think the second solution is much preferable to the first, unless we can’t achieve it for some reason. The first could be a stopgap temporary solution, better than a hang and arguably better than the incorrect behavior before #52706, but it still kind of sucks as I think we would agree that it’s not our intended final design. Ideally we can just go straight to the second solution and skip over any stopgaps like the exception or a revert.

@dygabo and @ShogunPanda, how feasible is the second solution? Is this something that we could potentially get a PR for in the next few days?