From 6c8cf59a7a2efbb2de4faae1fd7c76f4699dc87b Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Mon, 5 May 2025 17:16:53 -0700 Subject: [PATCH 1/4] Add the JIT to What's New and update some related docs --- Doc/using/configure.rst | 25 +++++++++++++++++-------- Doc/whatsnew/3.14.rst | 28 ++++++++++++++++++++++++++++ Tools/jit/README.md | 6 +++--- 3 files changed, 48 insertions(+), 11 deletions(-) diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 3d3776acbc7bea..7aa246aafe3064 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -302,14 +302,23 @@ General Options .. option:: --enable-experimental-jit=[no|yes|yes-off|interpreter] - Indicate how to integrate the :ref:`JIT compiler `. - - * ``no`` - build the interpreter without the JIT. - * ``yes`` - build the interpreter with the JIT. - * ``yes-off`` - build the interpreter with the JIT but disable it by default. - * ``interpreter`` - build the interpreter without the JIT, but with the tier 2 enabled interpreter. - - By convention, ``--enable-experimental-jit`` is a shorthand for ``--enable-experimental-jit=yes``. + Indicate how to integrate the :ref:`experimental just-in-time compiler + `. + + * ``no``: Don't build the JIT. + * ``yes``: Enable the JIT. To disable it at runtime, set the environment + variable :envvar:`PYTHON_JIT=0 `. + * ``yes-off``: Build the JIT, but disable it by default. To enable it at + runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. + * ``interpreter``: Enable the "JIT interpreter" (only useful for those + debugging the JIT itself). To disable the JIT at runtime, set the + environment variable :envvar:`PYTHON_JIT=0 `. + + By convention, ``--enable-experimental-jit`` is shorthand for + ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the + default behavior if the option is not provided. See + :file:`Tools/jit/README.md` for more information, including how to install + the necessary build-time dependencies. .. note:: diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index c35e4365de025b..8995c6068ebe5f 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -77,6 +77,7 @@ Summary -- release highlights :ref:`argparse `, :ref:`json ` and :ref:`calendar ` CLIs +* :ref:`An experimental just-in-time compiler ` Incompatible changes @@ -715,6 +716,33 @@ in the :envvar:`PYTHONSTARTUP` script. (Contributed by Ɓukasz Langa in :gh:`131507`.) +.. _whatsnew314-jit-compiler: + +An experimental just-in-time compiler +------------------------------------- + +Official macOS and Windows release binaries now include an *experimental* JIT +compiler. Although it is **not** recommended for production use, it can be +enabled by setting the :envvar:`PYTHON_JIT` environment variable. Downstream +source builds can be configured with :option:`--enable-experimental-jit=yes-off` +for similar behavior. + +This is still early, foundational work; the performance impact of enabling the +JIT typically ranges from 10% slower to 20% faster, depending on workload. The +most significant missing functionality at this time is the inability for native +debuggers and profilers like ``gdb`` and ``perf`` to unwind through JIT frames +(Python debuggers and profilers, like :mod:`pdb` or :mod:`profile`, work without +modification). Free-threaded builds do not support JIT compilation. + +:func:`sys._jit.is_available` can be used to determine if your Python executable +supports JIT compilation. To tell if JIT compilation has been enabled for the +current process, use :func:`sys._jit.is_enabled`. + +Please report any bugs or major performance regressions that you encounter! + +.. seealso:: :pep:`744` + + Other language changes ====================== diff --git a/Tools/jit/README.md b/Tools/jit/README.md index ff4b3964e65bad..8e817574b4d72b 100644 --- a/Tools/jit/README.md +++ b/Tools/jit/README.md @@ -54,13 +54,13 @@ choco install llvm --version=19.1.0 ## Building -For `PCbuild`-based builds, pass the new `--experimental-jit` option to `build.bat`. +For `PCbuild`-based builds, pass the `--experimental-jit` option to `build.bat`. -For all other builds, pass the new `--enable-experimental-jit` option to `configure`. +For all other builds, pass the `--enable-experimental-jit` option to `configure`. Otherwise, just configure and build as you normally would. Cross-compiling "just works", since the JIT is built for the host platform. -The JIT can also be enabled or disabled using the `PYTHON_JIT` environment variable, even on builds where it is enabled or disabled by default. More details about configuring CPython with the JIT and optional values for `--enable-experimental-jit` can be found [here](https://docs.python.org/dev/whatsnew/3.13.html#experimental-jit-compiler). +The JIT can also be enabled or disabled using the `PYTHON_JIT` environment variable, even on builds where it is enabled or disabled by default. More details about configuring CPython with the JIT and optional values for `--enable-experimental-jit` can be found [here](https://docs.python.org/dev/using/configure.html#cmdoption-enable-experimental-jit). [^pep-744]: [PEP 744](https://peps.python.org/pep-0744/) From 6ec0bad0a2154a218a9b3a68b453ff568b3ebd5b Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Mon, 5 May 2025 17:32:02 -0700 Subject: [PATCH 2/4] Clean up the diff --- Doc/using/configure.rst | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 7aa246aafe3064..4f9da89f89c9ed 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -302,23 +302,14 @@ General Options .. option:: --enable-experimental-jit=[no|yes|yes-off|interpreter] - Indicate how to integrate the :ref:`experimental just-in-time compiler - `. + Indicate how to integrate the :ref:`experimental just-in-time compiler `. * ``no``: Don't build the JIT. - * ``yes``: Enable the JIT. To disable it at runtime, set the environment - variable :envvar:`PYTHON_JIT=0 `. - * ``yes-off``: Build the JIT, but disable it by default. To enable it at - runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. - * ``interpreter``: Enable the "JIT interpreter" (only useful for those - debugging the JIT itself). To disable the JIT at runtime, set the - environment variable :envvar:`PYTHON_JIT=0 `. - - By convention, ``--enable-experimental-jit`` is shorthand for - ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the - default behavior if the option is not provided. See - :file:`Tools/jit/README.md` for more information, including how to install - the necessary build-time dependencies. + * ``yes``: Enable the JIT. To disable it at runtime, set the environment variable :envvar:`PYTHON_JIT=0 `. + * ``yes-off``: Build the JIT, but disable it by default. To enable it at runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. + * ``interpreter``: Enable the "JIT interpreter" (only useful for those debugging the JIT itself). To disable the JIT at runtime, set the environment variable :envvar:`PYTHON_JIT=0 `. + + By convention, ``--enable-experimental-jit`` is shorthand for ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the default behavior if the option is not provided. See :file:`Tools/jit/README.md` for more information, including how to install the necessary build-time dependencies. .. note:: From 470c7116f1f7222d0f0175a0228709297d616fc3 Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Mon, 5 May 2025 19:40:01 -0700 Subject: [PATCH 3/4] Wrap lines --- Doc/using/configure.rst | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 4f9da89f89c9ed..49d00d2f7a7047 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -305,11 +305,19 @@ General Options Indicate how to integrate the :ref:`experimental just-in-time compiler `. * ``no``: Don't build the JIT. - * ``yes``: Enable the JIT. To disable it at runtime, set the environment variable :envvar:`PYTHON_JIT=0 `. - * ``yes-off``: Build the JIT, but disable it by default. To enable it at runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. - * ``interpreter``: Enable the "JIT interpreter" (only useful for those debugging the JIT itself). To disable the JIT at runtime, set the environment variable :envvar:`PYTHON_JIT=0 `. - - By convention, ``--enable-experimental-jit`` is shorthand for ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the default behavior if the option is not provided. See :file:`Tools/jit/README.md` for more information, including how to install the necessary build-time dependencies. + * ``yes``: Enable the JIT. To disable it at runtime, set the environment + variable :envvar:`PYTHON_JIT=0 `. + * ``yes-off``: Build the JIT, but disable it by default. To enable it at + runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. + * ``interpreter``: Enable the "JIT interpreter" (only useful for those + debugging the JIT itself). To disable the JIT at runtime, set the + environment variable :envvar:`PYTHON_JIT=0 `. + + By convention, ``--enable-experimental-jit`` is shorthand for + ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the + default behavior if the option is not provided. See + :file:`Tools/jit/README.md` for more information, including how to install + the necessary build-time dependencies. .. note:: From a476d8ea0b22f40aceced2775203c2e4df3b2501 Mon Sep 17 00:00:00 2001 From: Brandt Bucher Date: Mon, 5 May 2025 20:00:21 -0700 Subject: [PATCH 4/4] Feedback from review --- Doc/using/configure.rst | 15 +++++++------- Doc/whatsnew/3.14.rst | 43 ++++++++++++++++++++++------------------- 2 files changed, 30 insertions(+), 28 deletions(-) diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 49d00d2f7a7047..b914d3397b6dda 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -310,14 +310,13 @@ General Options * ``yes-off``: Build the JIT, but disable it by default. To enable it at runtime, set the environment variable :envvar:`PYTHON_JIT=1 `. * ``interpreter``: Enable the "JIT interpreter" (only useful for those - debugging the JIT itself). To disable the JIT at runtime, set the - environment variable :envvar:`PYTHON_JIT=0 `. - - By convention, ``--enable-experimental-jit`` is shorthand for - ``--enable-experimental-jit=yes``. ``--enable-experimental-jit=no`` is the - default behavior if the option is not provided. See - :file:`Tools/jit/README.md` for more information, including how to install - the necessary build-time dependencies. + debugging the JIT itself). To disable it at runtime, set the environment + variable :envvar:`PYTHON_JIT=0 `. + + ``--enable-experimental-jit=no`` is the default behavior if the option is not + provided, and ``--enable-experimental-jit`` is shorthand for + ``--enable-experimental-jit=yes``. See :file:`Tools/jit/README.md` for more + information, including how to install the necessary build-time dependencies. .. note:: diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 8995c6068ebe5f..7b5f6cc1efdf0f 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -77,7 +77,7 @@ Summary -- release highlights :ref:`argparse `, :ref:`json ` and :ref:`calendar ` CLIs -* :ref:`An experimental just-in-time compiler ` +* :ref:`Binary releases for the experimental just-in-time compiler ` Incompatible changes @@ -718,25 +718,28 @@ in the :envvar:`PYTHONSTARTUP` script. .. _whatsnew314-jit-compiler: -An experimental just-in-time compiler -------------------------------------- - -Official macOS and Windows release binaries now include an *experimental* JIT -compiler. Although it is **not** recommended for production use, it can be -enabled by setting the :envvar:`PYTHON_JIT` environment variable. Downstream -source builds can be configured with :option:`--enable-experimental-jit=yes-off` -for similar behavior. - -This is still early, foundational work; the performance impact of enabling the -JIT typically ranges from 10% slower to 20% faster, depending on workload. The -most significant missing functionality at this time is the inability for native -debuggers and profilers like ``gdb`` and ``perf`` to unwind through JIT frames -(Python debuggers and profilers, like :mod:`pdb` or :mod:`profile`, work without -modification). Free-threaded builds do not support JIT compilation. - -:func:`sys._jit.is_available` can be used to determine if your Python executable -supports JIT compilation. To tell if JIT compilation has been enabled for the -current process, use :func:`sys._jit.is_enabled`. +Binary releases for the experimental just-in-time compiler +---------------------------------------------------------- + +The official macOS and Windows release binaries now include an *experimental* +just-in-time (JIT) compiler. Although it is **not** recommended for production +use, it can be tested by setting :envvar:`PYTHON_JIT=1 ` as an +environment variable. Downstream source builds and redistributors can use the +:option:`--enable-experimental-jit=yes-off` configuration option for similar +behavior. + +The JIT is at an early stage and still in active development. As such, the +typical performance impact of enabling it can range from 10% slower to 20% +faster, depending on workload. To aid in testing and evaluation, a set of +introspection functions has been provided in the :data:`sys._jit` namespace. +:func:`sys._jit.is_available` can be used to determine if the current executable +supports JIT compilation, while :func:`sys._jit.is_enabled` can be used to tell +if JIT compilation has been enabled for the current process. + +Currently, the most significant missing functionality is that native debuggers +and profilers like ``gdb`` and ``perf`` are unable to unwind through JIT frames +(Python debuggers and profilers, like :mod:`pdb` or :mod:`profile`, continue to +work without modification). Free-threaded builds do not support JIT compilation. Please report any bugs or major performance regressions that you encounter!