Segfault or abort in free-threaded build calling methods from exception in threads

[devdanzin]

Crash report

What happened?

In a no-gil build with PYTHON_GIL=0, it's possible to get the interpreter to segfault or abort with the message Fatal Python error: PyMutex_Unlock: unlocking mutex that is not locked by running the following code:

from threading import Thread
import email.errors

alive = []
for x in range(100):
    obj = email.errors.InvalidMultipartContentTransferEncodingDefect()

    alive.append(Thread(target=obj.__getstate__, args=()))
    alive.append(Thread(target=obj.__repr__, args=()))
    for x in range(20):
        obj.__init__(["a" * (2 ** 16)])
    alive.append(Thread(target=obj.__init__, args=(["a" * (2 ** 16)],)))

for obj in alive:
    try:
        print('START', obj)
        obj.start()
    except Exception:
        pass

Somehow it seems very hard to trigger a segfault under gdb, so here's the backtrace from the abort:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140736531846720) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140736531846720) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140736531846720, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7ce0476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7cc67f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x0000555555944db0 in fatal_error_exit (status=<optimized out>) at Python/pylifecycle.c:3059
#6  0x00005555559519fb in fatal_error (fd=2, header=header@entry=1,
    prefix=prefix@entry=0x555555b0bb20 <__func__.0> "PyMutex_Unlock",
    msg=msg@entry=0x555555b0b950 "unlocking mutex that is not locked", status=status@entry=-1)
    at Python/pylifecycle.c:3275
#7  0x0000555555951a6d in _Py_FatalErrorFunc (
    func=func@entry=0x555555b0bb20 <__func__.0> "PyMutex_Unlock",
    msg=msg@entry=0x555555b0b950 "unlocking mutex that is not locked") at Python/pylifecycle.c:3291
#8  0x000055555592a36f in PyMutex_Unlock (m=<optimized out>) at Python/lock.c:615
#9  0x00005555556c6e23 in _PyMutex_Unlock (m=<optimized out>) at ./Include/cpython/lock.h:60
#10 _PyCriticalSection_End (c=0x7fffc6fcc5f0) at ./Include/internal/pycore_critical_section.h:148
#11 list_repr (self=<optimized out>) at Objects/listobject.c:581
#12 0x0000555555709951 in PyObject_Repr (
    v=[<unknown at remote 0xdddddddddddddddd>, <unknown at remote 0xdddddddddddddddd>, <unknown at remote 0xdddddddddddddddd>, <unknown at remote 0xdddddddddddddddd>]) at Objects/object.c:737
#13 0x00005555557accbf in unicode_fromformat_arg (writer=writer@entry=0x7fffc6fcc750,
    f=0x555555a590bc "R)", f@entry=0x555555a590bb "%R)", vargs=vargs@entry=0x7fffc6fcc700)
    at Objects/unicodeobject.c:3048
#14 0x00005555557ad0dd in unicode_from_format (writer=writer@entry=0x7fffc6fcc750,
    format=format@entry=0x555555a590b8 "%s(%R)", vargs=vargs@entry=0x7fffc6fcc7c0)
    at Objects/unicodeobject.c:3167
#15 0x00005555557ad287 in PyUnicode_FromFormatV (format=0x555555a590b8 "%s(%R)",
    vargs=vargs@entry=0x7fffc6fcc7c0) at Objects/unicodeobject.c:3201
#16 0x00005555557ad363 in PyUnicode_FromFormat (format=format@entry=0x555555a590b8 "%s(%R)") at Objects/unicodeobject.c:3215
#17 0x00005555556986fb in BaseException_repr (self=self@entry=0x200005d2aa0) at Objects/exceptions.c:170
#18 0x000055555574c3f3 in wrap_unaryfunc (self=<InvalidMultipartContentTransferEncodingDefect(line=['aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...(truncated), args=<optimized out>, wrapped=0x555555698672 <BaseException_repr>) at Objects/typeobject.c:8947
#19 0x0000555555691ce5 in wrapperdescr_raw_call (kwds={}, args=(), self=<optimized out>, descr=<optimized out>) at Objects/descrobject.c:531
#20 wrapper_call (self=self@entry=<method-wrapper '__repr__' of InvalidMultipartContentTransferEncodingDefect object at 0x200005d2aa0>, args=args@entry=(), kwds=kwds@entry={}) at Objects/descrobject.c:1438
#21 0x000055555567f6ef in _PyObject_Call (tstate=0x555555dd7700, callable=callable@entry=<method-wrapper '__repr__' of InvalidMultipartContentTransferEncodingDefect object at 0x200005d2aa0>, args=args@entry=(), kwargs=kwargs@entry={}) at Objects/call.c:361
#22 0x000055555567f811 in PyObject_Call (callable=callable@entry=<method-wrapper '__repr__' of InvalidMultipartContentTransferEncodingDefect object at 0x200005d2aa0>, args=args@entry=(), kwargs=kwargs@entry={}) at Objects/call.c:373

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a2+ experimental free-threading build (heads/main-dirty:a13e94d84bf, Nov 23 2024, 07:16:19) [GCC 11.4.0]

[colesbury]

    alive.append(Thread(target=obj.__init__, args=(["a" * (2 ** 16)],)))

We are not currently planning on supporting concurrent re-initializations by calling __init__() on the same object. Would it be possible to exclude that pattern from your fuzzing?

[devdanzin]

Would it be possible to exclude that pattern from your fuzzing?

Sure, will do. Thanks for the heads up!