gh-125996: fix thread safety of ordered dict

[kumaraditya303]

• Issue: nogil segmentation fault on ordered dict operations #125996

[colesbury]

Would you please also provide some timing measurements for basic operations?

[colesbury]

_PyDict_Clear_LockHeld

[kumaraditya303]

_PyDict_Clear_LockHeld currently expects the critical section to held on the object but in tp_clear it is not held so it gives assertion failure, should I remove the assertion and use it instead?

The tp_clear of dict also currently uses PyDict_Clear and not _PyDict_Clear_LockHeld for likely similar reason I think.

cpython/Objects/dictobject.c

Lines 4647 to 4652 in 5de7e3f

	static int
	dict_tp_clear(PyObject *op)
	{
	PyDict_Clear(op);
	return 0;
	}

[colesbury]

If we need to hold a lock for the iterator, we can keep the Py_CLEAR(di->di_odict);

[kumaraditya303]

I have changed it now to first acquire the critical section on the iterator and also clears the dict when done, it still acquires the critical section on odict when looking up the next key.

[kumaraditya303]

Would you please also provide some timing measurements for basic operations?

I added some benchmark for get/set operation to ftscaling and benchmarked it:

index 1a59e25189d..20f15200051 100644
--- a/Tools/ftscalingbench/ftscalingbench.py
+++ b/Tools/ftscalingbench/ftscalingbench.py
@@ -28,6 +28,7 @@
 import threading
 import time
 from operator import methodcaller
+from collections import OrderedDict
 
 # The iterations in individual benchmarks are scaled by this factor.
 WORK_SCALE = 100
@@ -43,6 +44,19 @@ def register_benchmark(func):
     ALL_BENCHMARKS[func.__name__] = func
     return func
 
+@register_benchmark
+def odict_get():
+    tab = OrderedDict((i, i) for i in range(100))
+    for i in range(1000 * WORK_SCALE):
+        tab.get(i % 100)
+
+@register_benchmark
+def odict_set():
+    tab = OrderedDict((i, i) for i in range(100))
+    for i in range(1000 * WORK_SCALE):
+        tab[i % 100] = i
+
+
 @register_benchmark
 def object_cfunction():
     accu = 0

Results:

❯ ./python.exe Tools/ftscalingbench/ftscalingbench.py odict_get odict_set
Running benchmarks with 10 threads
odict_get                  4.7x faster
odict_set                  4.4x faster