Assertion failure when interrupting active threads for subinterpreters

[neonene]

Crash report

What happened?

The following code runs a subinterpreter in a new non-daemon thread, where CTRL+C+C interruption causes an assertion failure on debug builds:

from concurrent import interpreters

def f():
    import time
    time.sleep(100)

if __name__ == '__main__':
    interp = interpreters.create()
    print('press CTRL^C twice:')
    t = interp.call_in_thread(f)
    t.join()

Assertion failed: !_PyInterpreterState_IsRunningMain(interp), file C:\rc\Python\pylifecycle.c, line 2516

The same assert can be triggered without keystrokes:

import threading
from concurrent import interpreters

interp = interpreters.create()
t = threading.Thread(target=interp.exec, daemon=True,
                     args=('import time; time.sleep(3)',))
t.start()

cc @ericsnowcurrently

CPython versions tested on:

3.14.0rc1, CPython main branch: 2a87af0

Operating systems tested on:

Windows

[ZeroIntensity]

I think this is probably a duplicate of #128639. The linked PR there should indeed fix these pesky automatic subinterpreter shutdown problems.

[neonene]

a duplicate of #128639

I mostly agree. 3.14.0rc1 reports an assertion failure at a different place (PyThreadState_Clear) when the pylifecycle.c patch of #134606 is applied:

<sys>:0: RuntimeWarning: remaining subinterpreters; destroy them with _interpreters.destroy()
Assertion failed: !_PyThreadState_IsRunningMain(tstate), file C:\rc\Python\pystate.c, line 1724

[ZeroIntensity]

Ok, let's deal with this after #134606 is merged (because I suspect any test case you come up with will be hit by issues from #128639 in addition to this one). I think the problem is likely that the thread is getting hung with the subinterpreter active, which _PyThreadState_DeleteList doesn't like.

[neonene]

Another reproducer:

from concurrent.futures import InterpreterPoolExecutor

def f():
    import time
    time.sleep(100)

if __name__ == "__main__":
    exe = InterpreterPoolExecutor()
    wait = True
    print(f'press CTRL^C {"twice" if wait else "once"}:')
    for _ in range(5):
        exe.submit(f)
    exe.shutdown(wait=wait)