dune exec --watch broken for long-running processes

[royneary]

Expected Behavior

For long-running processes like webservers, I'm normally starting dune exec --watch binary_name. Whenever I change a source code file, dune kills the running process, recompiles it and starts it again. This works with dune 3.19.1.

Actual Behavior

Since upgrading to dune 3.20.1, when changing the source code, the long-running process is not killed anymore. Recompilation only starts when the process exits on its own or I kill it manually.

Reproduction

1. create a dune project with the following code in bin/main.ml:

   let () =
     for i = 20 downto 0 do
       Unix.sleep 1;
       print_endline (string_of_int i);
     done
   

   and the following bin/dune:

   (executable
     (public_name countdown)
     (name main)
     (libraries unix))
   

2. start dune exec --watch countdown

3. change bin/main.ml

The countdown continues until it reaches 0. Then recompilation starts and the process is started again.

Specifications

• Version of dune (output of dune --version): 3.20.1
• Version of ocaml (output of ocamlc --version): 5.3.0
• Operating system (distribution and version): NixOS 25.05

[Alizter]

We are sending SIGTERM to the subprocess and I am wondering if this is the cause of the problem you are observing. When you kill it manually are you sending SIGTERM or SIGKILL? If your application terminates when hit with both, then there must be something going wrong in dune.

[Alizter]

It's likely #11841 that is the cause of this issue.

[royneary]

Thanks for the quick reply. I'm sending SIGTERM to kill the process using htop and it terminates immediately.

[Alizter]

I've managed to bisect the issue to 120e8ac. This commit was intending to fix a different issue with dune exec -w by attempting to revert to the previous spawn behaviour. It looks like an unintended side effect was this issue.

[Alizter]

The bug is subtle, but I think I understand the issue.

Before 120e8ac we were creating a new process group for the spawned executable. This caused various issues such as the tty interacting poorly. To fix this we reverted back to inheriting the same process group as dune.

This however causes another issue. Dune kills processes by killing their process group ids. However this assumes that the pid is the process group id, which is not true in the case of the process we spawn using dune exec.

I'm not entirely sure if and how this can be fixed yet, but I thought I would share my understanding until I get there.

[Alizter]

@royneary I've created a fix in #12360. I would appreciate if you could give it a try. Let me know if you need any assistance building dune.

[royneary]

I think #12360 fixes it!

Here's what I did:

$ git clone -b dune-exec-watch-server https://github.com/Alizter/dune.git
$ cd dune
$ make release # I hit https://github.com/ocaml/dune/issues/12075 at first, but it worked using a fresh opam switch
$ ./_build/default/bin/dune.exe --version
2.7.0-7301-ge61a2dd-dirty

I don't understand why the version number starts with 2.7.0, but I assume that is expected.

If I use that dune.exe binary in exec --watch mode, long-running processes are killed again.