Stub binary doesn't support `sys.executable` spawn/multiprocessing

[freakboy3742]

Describe the bug

The stub binary used to start an app is an effective replacement for the python binary; however, it isn't a perfect replacements. multiprocessing in spawn mode will try to invoke sys.executable to spawn a child; it's also common to use subprocess or os.spawn to try and start sys.exectuable to run other Python code as a subprocess. However, the stub binary doesn't support this kind of execution.

Steps to reproduce

1. Generate a stub macOS app
2. Add a call to call multiprocessing.Process() and start it with .start()

The subprocess will call the parent process entry point, and fail.

Expected behavior

The function targeted by Process() should be invoked with the provided arguments.

Screenshots

No response

Environment

• Operating System: All macOS
• Python version: All
• Software versions:
  • Briefcase: 0.3.12

Logs

n/a

Additional context

See beeware/briefcase-macOS-app-template#7 for the original report on macOS; the fault is with the stub binary, so the fix needs to be on the Xcode template. Any fix here will be inherited by the app template.

One possible solution would be to add some logic to the stub binary to inspect to see if the parent process is "self"; if so, the execution of the app should be modified to behave as if it were a normal Python binary.

Any solution should also be ported to Linux (beeware/briefcase-linux-flatpak-template#10) and Windows (beeware/briefcase-windows-VisualStudio-template#8).

[hartwoolery]

I'd like to help fix this issue if possible - not sure where needs to be fixed. Every time I launch my app it's spawning infinite instances and its difficult to debug with lots of code and having to restart my machine every time I test it.

[freakboy3742]

@hartwoolery I don't have a specific fix in mind; this is going to require some investigation, elaboration and discussion.

There's two pieces interacting to cause undesirable behavior:

The first is the stub binary. The template contains a main.c file that is compiled and used as the stub binary. It's effectively the same as python.exe - except that it hard-codes the python program that is going to be executed. However, it's not identical to python.exe in some behaviours - and the behavior under spawn is one of those behaviours.

The second is multiprocessing itself. The launch mechanism uses spawn or fork to create child processes. Ideally, we won't have to make any changes here... but we might not be able to avoid it. At the very least, this is the logic that you can reproduce for testing purposes.

So - we need to modify the stub binary (and maybe the multiprocessing library) so that the spawn/fork behavior doesn't cause the failure cycle that you're seeing. The question is how.

[hartwoolery]

Thanks for the information - I was able to narrow down my issue by trial and error to a glfw import (an OpenGL lib) which was causing windows to respawn. I'll keep my eye on this for any updates, but sounds non-trivial!