Skip to content

COMPAT: Fix warning with numba >= 0.58.0 #55327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 11, 2024
Merged

COMPAT: Fix warning with numba >= 0.58.0 #55327

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9343eca
COMPAT: Fix warning with numba >= 0.58.0
lithomas1 Sep 29, 2023
aa60545
Update numba_.py
lithomas1 Oct 2, 2023
e6995b6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 2, 2023
891c05b
Merge branch 'main' into numba-0.58-warnings
lithomas1 Oct 12, 2023
1f7b25e
Merge branch 'main' into numba-0.58-warnings
lithomas1 Dec 28, 2023
16a453d
add a test
lithomas1 Dec 28, 2023
c8bb008
skip if no numba
lithomas1 Dec 29, 2023
df4f48a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 29, 2023
860633b
Update v2.2.0.rst
lithomas1 Jan 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/source/whatsnew/v2.2.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -727,6 +727,11 @@ Performance improvements

Bug fixes
~~~~~~~~~
- Bug in :class:`AbstractHolidayCalendar` where timezone data was not propagated when computing holiday observances (:issue:`54580`)
- Bug in :class:`pandas.core.window.Rolling` where duplicate datetimelike indexes are treated as consecutive rather than equal with ``closed='left'`` and ``closed='neither'`` (:issue:`20712`)
- Bug in :meth:`DataFrame.apply` where passing ``raw=True`` ignored ``args`` passed to the applied function (:issue:`55009`)
- Bug in :meth:`pandas.read_excel` with a ODS file without cached formatted cell for float values (:issue:`55219`)
- Fixed a spurious deprecation warning from ``numba`` >= 0.58.0 when passing a numpy ufunc in :class:`pandas.core.window.Rolling.apply` with ``engine="numba"`` (:issue:`55247`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should belong in the Other section, and the other entries have already been moved I believe


Categorical
^^^^^^^^^^^
Expand Down
9 changes: 9 additions & 0 deletions pandas/core/util/numba_.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
"""Common utilities for Numba operations"""
from __future__ import annotations

import types
from typing import (
TYPE_CHECKING,
Callable,
)

import numpy as np

from pandas.compat._optional import import_optional_dependency
from pandas.errors import NumbaUtilError

Expand Down Expand Up @@ -83,6 +86,12 @@ def jit_user_function(func: Callable) -> Callable:
if numba.extending.is_jitted(func):
# Don't jit a user passed jitted function
numba_func = func
elif getattr(np, func.__name__, False) is func or isinstance(
func, types.BuiltinFunctionType
):
# Not necessary to jit builtins or np functions
# This will mess up register_jitable
numba_func = func
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor thing... I think this breaks the implied contract in the function name jit_user_function, because this is returning a function that is not jit-compiled (and has also probably not come from user derived source!). I do however think it is likely restore the functionality of being able to pass a NumPy or builtin function to rolling.apply xref: #55247 (comment).

Perhaps it would be good to add a unit test that exercises this path with view of preventing a similar issue in future (assuming there isn't one present now?).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the slow reply here.

I think this should be fine - the return type in the docstring says it can return a function that is JITable by numba.

(In the pandas code, I also don't think there's anywhere where we directly apply the jitted user function on data (we always pass it to another jitted function that calls the output from jit_user_function).)

Will add a test for this, though.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking this and adding the test.

else:
numba_func = numba.extending.register_jitable(func)

Expand Down
7 changes: 7 additions & 0 deletions pandas/tests/window/test_numba.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,3 +446,10 @@ def test_table_method_ewm(self, data, method, axis, nogil, parallel, nopython):
engine_kwargs=engine_kwargs, engine="numba"
)
tm.assert_frame_equal(result, expected)


@td.skip_if_no("numba")
def test_npfunc_no_warnings():
df = DataFrame({"col1": [1, 2, 3, 4, 5]})
with tm.assert_produces_warning(False):
df.col1.rolling(2).apply(np.prod, raw=True, engine="numba")