GH-46728: [Python] Skip test_gdb.py tests if PyArrow wasn't built debug #46755

dinse · 2025-06-09T22:31:43Z

Rationale for this change

As mentioned in #46728, if Arrow C++ was built debug, and PyArrow wasn't, test_gdb.py runs tests that fail.

What changes are included in this PR?

The CMAKE_BUILD_TYPE environment variable is propagated from build into PyArrow, where it's checked to skip unit tests.

Are these changes tested?

Yes. I have built PyArrow in release, debug, and relwithdebinfo and observed the new behavior. Because CMakeLists.txt was changed, I built PyArrow twice via setup.py and pip install, and checked the new function.

Are there any user-facing changes?

Devs may skip unit tests that would fail. PyArrow now has build_info() with information about the build type.

GitHub Issue: [Python] Skip test_gdb.py tests if PyArrow was built with non-debug build #46728

github-actions · 2025-06-09T22:32:08Z

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

github-actions · 2025-06-09T23:49:03Z

⚠️ GitHub issue #46728 has been automatically assigned in GitHub to PR creator.

pitrou

Thanks for doing this. Here is a bunch of suggestions, but obviously this is a good fix.

pitrou · 2025-06-10T07:31:40Z

python/pyarrow/lib.pyx

+    Returns the PyArrow build type (debug, minsizerel, release, 
+    relwithdebinfo). The default build type is release, regardless of if C++ 
+    was built in debug mode.


Can we keep using the summary line + longer description convention?

pitrou · 2025-06-10T07:32:50Z

python/pyarrow/lib.pyx

@@ -96,6 +96,15 @@ def is_threading_enabled() -> bool:
    return libarrow_python.IsThreadingEnabled()


+def get_pybuild_type() -> str:


The function name is awkward and restrictive. Perhaps we want a function def build_info() -> dict that returns various pieces of information about the build and, more importantly, that we can extend in the future?

Ok, pa.build_info() now returns a namedtuple.

pitrou · 2025-06-10T07:33:34Z

python/CMakeLists.txt

+
+# Write out compile-time configuration constants
+string(TOLOWER ${CMAKE_BUILD_TYPE} LOWERCASE_PYBUILD_TYPE)
+configure_file("${PYARROW_CPP_SOURCE_DIR}/pybuild_type.h.cmake" "${PYARROW_CPP_SOURCE_DIR}/pybuild_type.h" ESCAPE_QUOTES)


Why not make this something like config.h.cmake? We may want to add other constants there in the future.

Ok, there is now config.h/.cc instead of doing this in helpers, and config_internal.h.cmake.

pitrou · 2025-06-10T07:34:40Z

python/pyarrow/src/arrow/python/helpers.cc

+namespace {
+  const std::string kPyBuildType = PYARROW_CYTHON_BUILD_TYPE;
+}
+
+std::string GetPyBuildType() {
+  return kPyBuildType;
+}


The separate constant seems a bit pointless to me? This will return a copy anyway.

Suggested change

namespace {

const std::string kPyBuildType = PYARROW_CYTHON_BUILD_TYPE;

}

std::string GetPyBuildType() {

return kPyBuildType;

}

std::string GetPyBuildType() {

return PYARROW_CYTHON_BUILD_TYPE;

}

pitrou · 2025-06-10T07:35:19Z

python/pyarrow/src/arrow/python/pybuild_type.h.cmake

+// specific language governing permissions and limitations
+// under the License.
+
+#define PYARROW_CYTHON_BUILD_TYPE "@LOWERCASE_PYBUILD_TYPE@"


I would simply call it PYARROW_BUILD_TYPE, because this is not related to Cython.

pitrou · 2025-06-10T07:35:52Z

python/pyarrow/tests/test_gdb.py

@@ -25,6 +25,7 @@
 import pytest

 import pyarrow as pa
+from pyarrow.lib import get_pybuild_type


You don't need to import it explicitly, just call pa.get_pybuild_type? (and make sure this is exposed in the top-level pyarrow namespace)

Fixed. Exposed pa.build_info().

pitrou · 2025-06-10T07:36:07Z

python/pyarrow/tests/test_misc.py

@@ -23,6 +23,7 @@

 import pyarrow as pa
 from pyarrow.lib import ArrowInvalid
+from pyarrow.lib import get_pybuild_type


pitrou · 2025-07-01T05:55:56Z

python/pyarrow/config.pxi

@@ -77,6 +78,27 @@ def runtime_info():
        detected_simd_level=frombytes(c_info.detected_simd_level))


+PythonBuildInfo = namedtuple(


Suggestion:

rename the BuildInfo class above to CppBuildInfo

rename this PythonBuildInfo class to BuildInfo

add the CppBuildInfo as an attribute of BuildInfo

Ok, did that.

pitrou · 2025-07-01T05:56:05Z

python/pyarrow/tests/test_misc.py

@@ -94,6 +94,10 @@ def test_build_info():
    assert pa.cpp_build_info.build_type in (
        'debug', 'release', 'minsizerel', 'relwithdebinfo')

+    assert isinstance(pa.build_info(), pa.PythonBuildInfo)


Ah, I realize that pa.cpp_build_info is a simple attribute, not a function, so perhaps pa.build_info should be made an attribute as well?

Ok, did so.

WillAyd · 2025-07-01T13:30:07Z

python/pyarrow/tests/test_gdb.py

@@ -195,6 +195,8 @@ def gdb():
 def gdb_arrow(gdb):
    if 'deb' not in pa.cpp_build_info.build_type:
        pytest.skip("Arrow C++ debug symbols not available")
+    if pa.build_info().build_type != 'debug':


Should we also allow relwithdebug as a value?

Someone would have to try it out, but I think relwithdebug might optimize local variables away.

With relwithdebinfo, less of the tests fail than release but several still fail.

pitrou

Thanks for the update! Still one issue remaining, otherwise I think we're good to go.

pitrou · 2025-07-09T12:02:31Z

python/pyarrow/includes/libarrow_python_config.pxd

@@ -0,0 +1,29 @@
+# Licensed to the Apache Software Foundation (ASF) under one


This file would be suitable if we actually had a libarrow_python_config shared library, but we don't, these APIs are part of libarrow_python. Therefore, these declarations should be moved to libarrow_python.pxd.

I am new to Cython and was getting ambiguous overload compilation errors from both GetBuildInfos with a variety of approaches. I did this dummy namespace pxd to address naming conflicts; the docs say

"There won’t be any actual [libarrow_python_config] module at run time, but that doesn’t matter; the [libarrow_python_config].pxd file has done its job of providing an additional namespace at compile time."

Anyways, I agree it makes sense to keep things in libarrow_python if possible, and I've since found a way to do that.

python/pyarrow/config.pxi

python/pyarrow/tests/test_misc.py

pitrou

+1, will merge if CI is green. Thank you @dinse .

… in non-release build

…uild type.

… review comments

dinse requested review from AlenkaF, raulcd and rok as code owners June 9, 2025 22:31

github-actions bot added Component: Python awaiting review Awaiting review labels Jun 9, 2025

dinse changed the title ~~Arrow 46728 - Skip test_gdb.py tests if PyArrow wasn't built debug~~ GH-46728: [Python] Skip test_gdb.py tests if PyArrow wasn't built debug Jun 9, 2025

pitrou requested changes Jun 10, 2025

View reviewed changes

github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jun 24, 2025

pitrou reviewed Jul 1, 2025

View reviewed changes

WillAyd reviewed Jul 1, 2025

View reviewed changes

dinse force-pushed the arrow-46728-gdb-skip branch from 2b22170 to 6990c34 Compare July 2, 2025 23:54

pitrou requested changes Jul 9, 2025

View reviewed changes

pitrou reviewed Jul 15, 2025

View reviewed changes

python/pyarrow/config.pxi Show resolved Hide resolved

pitrou reviewed Jul 15, 2025

View reviewed changes

python/pyarrow/tests/test_misc.py Show resolved Hide resolved

pitrou approved these changes Jul 15, 2025

View reviewed changes

dinse and others added 8 commits July 15, 2025 17:14

Issue apache#46728 - propagating PyArrow build type to skip GDB tests…

4a1a1f3

… in non-release build

Issue apache#46728 - cleaning up print debugging and adding missing b…

fc8715f

…uild type.

Issue apache#46728 - Making docstring adhere to 80 char width.

3b64f48

Issue apache#46728 - Switching the python build-info to a namedtuple.

16b4fe5

Moving the PyArrow and libarrow cpp build info together based on code…

c53f9c1

… review comments

Issue apache#46728 - Fixing code review comments

ec2a284

Apply suggestions from code review

c7181c8

Fix test failure

f9aa2da

pitrou force-pushed the arrow-46728-gdb-skip branch from 6ddbb1a to f9aa2da Compare July 15, 2025 15:16

pitrou merged commit 6c9e30b into apache:main Jul 15, 2025
12 of 13 checks passed

pitrou removed the awaiting committer review Awaiting committer review label Jul 15, 2025

pitrou mentioned this pull request Jul 15, 2025

[Python] Skip test_gdb.py tests if PyArrow was built with non-debug build #46728

Closed

dinse deleted the arrow-46728-gdb-skip branch July 15, 2025 23:16

		@@ -96,6 +96,15 @@ def is_threading_enabled() -> bool:
		return libarrow_python.IsThreadingEnabled()


		def get_pybuild_type() -> str:

		@@ -77,6 +78,27 @@ def runtime_info():
		detected_simd_level=frombytes(c_info.detected_simd_level))


		PythonBuildInfo = namedtuple(

		@@ -0,0 +1,29 @@
		# Licensed to the Apache Software Foundation (ASF) under one

GH-46728: [Python] Skip test_gdb.py tests if PyArrow wasn't built debug #46755

GH-46728: [Python] Skip test_gdb.py tests if PyArrow wasn't built debug #46755

Uh oh!

Conversation

dinse commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions bot commented Jun 9, 2025

Uh oh!

github-actions bot commented Jun 9, 2025

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dinse Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dinse commented Jun 9, 2025 •

edited

Loading

dinse Jul 1, 2025 •

edited

Loading