Skip to content

change pandas engine to be numpy>2 compat #1690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cosmicBboy merged 27 commits into from
Jun 20, 2024
Merged

change pandas engine to be numpy>2 compat #1690

cosmicBboy merged 27 commits into from
Jun 20, 2024

Conversation

@cosmicBboy
Copy link
Collaborator

@cosmicBboy cosmicBboy commented Jun 17, 2024 •
edited
Loading

fixes #1656

This PR:

  • Updates the pandas/numpy type engine to stop using deprecated numpy types
  • Updates CI to run the extras packages in a separate job
  • Moves extra package (pyspark, dask, modin, etc) out of the top-level module-loading execution path of pandera
  • Pins numpy<2 in extra package CI
  • Adds a temporary numpy2 patch until dependent packages (pyspark, modin, etc) support numpy2
  • Disables decimal type test for modin
  • Pins numpy<2 in readthedocs config yaml

@cosmicBboy cosmicBboy mentioned this pull request Jun 17, 2024
Closed
3 tasks
@cosmicBboy
update
9105027 
Signed-off-by: Niels Bantilan <[email protected]>
@cosmicBboy
debug
0637cf5 
Signed-off-by: Niels Bantilan <[email protected]>
@cosmicBboy
update requirements
bdbe2e0 
Signed-off-by: Niels Bantilan <[email protected]>
@cosmicBboy
update requirements, only use numpy2 for latest pandas
bcd9198 
Signed-off-by: Niels Bantilan <[email protected]>
@cosmicBboy
move pyspark config
63838ad 
Signed-off-by: Niels Bantilan <[email protected]>
@cosmicBboy
remove typing from root __init__
03a8b25 
Signed-off-by: Niels Bantilan <[email protected]>
@cmarshak
Copy link

cmarshak commented Jun 19, 2024

Thank you for fixing this!

cosmicBboy added 17 commits June 19, 2024 08:10
@cosmicBboy
[wip] try patching numpy2 with NaN
c4e06ee 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
[wip] remove pyspark import from typing.pandas
c809bed 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
isolate dask, modin, pyspark, etc imports, update ci
2aca4be 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
update numpy patch
515d65f 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
fix ci numpy pin
e6cd14a 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
update ci
11f2aee 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
update
63d6203 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
update
8230cc3 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
00f568e 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
d51a372 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
f72c0a3 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
update ci test for mypy
877059a 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
add numpy patch to root pandera
e93842b 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
cdf1a14 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
527f79a 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
debug
5347a5a 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy
skip modin test for decimals, remove pandas2.0.3
c7a9b29 
Signed-off-by: cosmicBboy <[email protected]>
@cosmicBboy cosmicBboy merged commit 47c236e into main Jun 20, 2024
@cosmicBboy cosmicBboy deleted the bugfix/1656 branch June 20, 2024 14:13
@cmarshak cmarshak mentioned this pull request Jun 20, 2024
Merged
@kr-hansen kr-hansen mentioned this pull request Jun 25, 2024
Closed
@cosmicBboy cosmicBboy mentioned this pull request Jul 17, 2024
Closed
@Girmii Girmii mentioned this pull request Dec 5, 2024
Closed
3 tasks
max-raphael pushed a commit to max-raphael/pandera that referenced this pull request Jan 24, 2025
@cosmicBboy @max-raphael
change pandas engine to be numpy>2 compat (unionai-oss#1690)
2a93b2a 
* change pandas engine to be numpy>2 compat

Signed-off-by: Niels Bantilan <[email protected]>

* remove numpy from mypy pre-commit type stubs

Signed-off-by: Niels Bantilan <[email protected]>

* install numpy<2.0.0 in readthedocs build

Signed-off-by: Niels Bantilan <[email protected]>

* remove check docs ci test

Signed-off-by: Niels Bantilan <[email protected]>

* update

Signed-off-by: Niels Bantilan <[email protected]>

* debug

Signed-off-by: Niels Bantilan <[email protected]>

* update requirements

Signed-off-by: Niels Bantilan <[email protected]>

* update requirements, only use numpy2 for latest pandas

Signed-off-by: Niels Bantilan <[email protected]>

* move pyspark config

Signed-off-by: Niels Bantilan <[email protected]>

* remove typing from root __init__

Signed-off-by: Niels Bantilan <[email protected]>

* [wip] try patching numpy2 with NaN

Signed-off-by: cosmicBboy <[email protected]>

* [wip] remove pyspark import from typing.pandas

Signed-off-by: cosmicBboy <[email protected]>

* isolate dask, modin, pyspark, etc imports, update ci

Signed-off-by: cosmicBboy <[email protected]>

* update numpy patch

Signed-off-by: cosmicBboy <[email protected]>

* fix ci numpy pin

Signed-off-by: cosmicBboy <[email protected]>

* update ci

Signed-off-by: cosmicBboy <[email protected]>

* update

Signed-off-by: cosmicBboy <[email protected]>

* update

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* update ci test for mypy

Signed-off-by: cosmicBboy <[email protected]>

* add numpy patch to root pandera

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* debug

Signed-off-by: cosmicBboy <[email protected]>

* skip modin test for decimals, remove pandas2.0.3

Signed-off-by: cosmicBboy <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: cosmicBboy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

pandera not compatible with numpy 2.0

3 participants

@cosmicBboy @cmarshak