Commit f9b0af5
bugfix: fix
* resolve 2035
Signed-off-by: Andrej Iring <[email protected]>
* change astype to apply
Signed-off-by: Andrej Iring <[email protected]>
* bugfix: PANDERA_VALIDATION_ENABLED=False should disable validation (#2028)
* bugfix: PANDERA_VALIDATION_ENABLED=False should disable validation
Signed-off-by: Niels Bantilan <[email protected]>
* fix tests
Signed-off-by: Niels Bantilan <[email protected]>
---------
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Expect Python slice index errors after Python 3.10 (#2033)
* Expect Python slice index errors with pandas again
Seen in https://github.com/unionai-oss/pandera/actions/runs/15526426636/job/43706977542?pr=2030
Signed-off-by: Deepyaman Datta <[email protected]>
* Expect Python slice index errors after Python 3.10
Signed-off-by: Deepyaman Datta <[email protected]>
* Fix the version check to not include Python 3.10.x
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* resolve failing pyspark tests
Signed-off-by: Andrej Iring <[email protected]>
* add new test
Signed-off-by: Andrej Iring <[email protected]>
* format code using black
Signed-off-by: Andrej Iring <[email protected]>
* Add `DataFrameModel`, `DataFrameSchema` for `ibis`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Refactor Ibis `DataFrameSchema` to extend pandas's
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add code for basic `Column`, and stub more modules
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add Ibis's parsing, validation, and error backends
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement stub to validate schema component checks
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement validation for floating types
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Inherit getter, only override setter, for `.dtype`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* implement column, add basic unit tests for Ibis data types
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add `ibis` extra and regenerate requirements files
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Re-enable Python equivalents for `int` and `float`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Do not test Ibis below py3.9, add DuckDB for tests (#1773)
* Restore accidentally-deleted use of "breakpoint()"
Signed-off-by: Deepyaman Datta <[email protected]>
* Swap `types-pkg_resources` with `types-setuptools` (#1779)
* Swap `types-pkg_resources` with `types-setuptools`
Signed-off-by: Deepyaman Datta <[email protected]>
* Update the expected error message to name variable
Signed-off-by: Deepyaman Datta <[email protected]>
* Fix expected outputs for pandas-stubs 2.2.2.240807
Signed-off-by: Deepyaman Datta <[email protected]>
* Update CI configuration to filter right pandas ver
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
* Do not test Ibis below py3.9, add DuckDB for tests
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Support specifying and validating optional columns (#1762)
* Test model schema equivalency with optional dtypes
Signed-off-by: Deepyaman Datta <[email protected]>
* Support specifying and validating optional columns
Signed-off-by: Deepyaman Datta <[email protected]>
* Add DuckDB for testing and regenerate requirements
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Register string datatype and update existing tests (#1766)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add missing `type` field for the `Int32` data type (#1771)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement minimal `coerce` and corresponding tests (#1772)
* Implement minimal `coerce` and corresponding tests
Signed-off-by: Deepyaman Datta <[email protected]>
* Do not exclude 3.9 in Ibis CI, and regenerate reqs
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Ibis check backend (#1831)
* [wip] add minimal ibis check backend implementation
Signed-off-by: cosmicBboy <[email protected]>
* support scalar, column, and table check output types
Signed-off-by: cosmicBboy <[email protected]>
* support scalar, column, and table check output types
Signed-off-by: cosmicBboy <[email protected]>
* Ibis check backend suggestions (#1855)
* Apply suggestions from code review
Signed-off-by: Deepyaman Datta <[email protected]>
* Fix row-order-dependent order by adding table cols
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
* fix lint
Signed-off-by: cosmicBboy <[email protected]>
* fix unit tests
Signed-off-by: cosmicBboy <[email protected]>
---------
Signed-off-by: cosmicBboy <[email protected]>
Signed-off-by: Deepyaman Datta <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Remove Ibis backend's use of `scalar_failure_case`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Update signature of `run_schema_components_checks`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Fix violations in `pandera/backends//container.py`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Remove use of multimethod library and align Polars
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* fix backend registration
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement minimal built-in checks for Ibis backend (#1885)
* Implement minimal built-in checks for Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
* Implement `Column` validation for the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
* Promote check object to table during preprocessing
Signed-off-by: Deepyaman Datta <[email protected]>
* Remove extraneous fixture for backend registration
Signed-off-by: Deepyaman Datta <[email protected]>
* Resolve lint (unused imports, undefined variables)
Signed-off-by: Deepyaman Datta <[email protected]>
* Partially standardize docstrings of builtin checks
Signed-off-by: Deepyaman Datta <[email protected]>
* Fix the `preprocess` docstrings copied from pandas
Signed-off-by: Deepyaman Datta <[email protected]>
* Format pandera/backends/ibis/checks.py using Black
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Update test to reflect scalar output since Ibis 10 (#1907)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Use `Union` type for `PysparkObject` to fix typing
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add Ibis with the DuckDB extra as a dev dependency
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Update/align noxfile.py and generated requirements
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Supress unused argument check for `data_container`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Include `polars` extra dep for testing Ibis dtypes
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Update check example to correctly process IbisData
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Execute validated schema to get the desired result
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement "ne" built-in check for the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement int/uint/float types, except for float16
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement timestamp type, and test built-in checks
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Support built-in checks for interval-typed columns
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Blacken pandera/engines/ibis_engine.py module code
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `dt.Date` type, and test built-in checks
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `dt.Time` type, and test built-in checks
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `gt` and `ge` check for the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Standardize docstrings, don't say "data container"
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `lt` and `le` check for the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Fix "form" to "from", and align docstring for test
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Apply suggestion from Copilot to fix error message
Co-authored-by: Copilot <[email protected]>
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Support table-level checks, including for built-in
Signed-off-by: Deepyaman Datta <[email protected]>
Co-authored-by: cosmicBboy <[email protected]>
Signed-off-by: cosmicBboy <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `is_in_range` check for the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `isin()` built-in check for Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Refactor duplicated code into `_across()` function
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `notin()` builtin check for Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement the `str_matches` built-in check on Ibis
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `str_contains` check on the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `str_startswith()` check on Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `str_endswith()` built-in check for Ibis
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement the `str_length` built-in check for Ibis
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement the unique_values_eq built-in Ibis check
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add entry for `ibis` to the backend support matrix
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Write detailed documentation on using Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Replace `ibis.expr.types` imports (where possible)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Implement `typing` module and decorators for Ibis
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Duplicate tests for Ibis decorators implementation
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Use `code-cell` over `testcode` to build Ibis docs
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Add Ibis docs to the supported libraries `toctree`
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Create element-wise checks automatically for users (#2043)
* Create element-wise checks automatically for users
Signed-off-by: Deepyaman Datta <[email protected]>
* Mark unsupported scenario with element-wise checks
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Remove `to_format` methods that don't work on Ibis (#2044)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Execute dataframe-level checks on the Ibis backend (#2041)
* Execute dataframe-level checks on the Ibis backend
Signed-off-by: Deepyaman Datta <[email protected]>
* Remove unnecessary, un-Pythonic array length check
Signed-off-by: Deepyaman Datta <[email protected]>
* Remove bit on separately running data-level checks
Signed-off-by: Deepyaman Datta <[email protected]>
---------
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* Mark unsupported scenario with element-wise checks (#2046)
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* handle dataframe-level failure cases: convert row to dict
Signed-off-by: cosmicBboy <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* bugfix/1927 (#2019)
* fix generating pydantic core schema for generic types
Signed-off-by: Jarek-Rolski <[email protected]>
* test pytest for typed generic DataFrame
Signed-off-by: Jarek-Rolski <[email protected]>
* fix issue with docs gitactions test
Signed-off-by: Jarek-Rolski <[email protected]>
* fix pytest test_typed_generic_dataframe
Signed-off-by: Jarek-Rolski <[email protected]>
* fix pytest test_typed_generic_dataframe
Signed-off-by: Jarek-Rolski <[email protected]>
* improve changes
Signed-off-by: Jarek-Rolski <[email protected]>
* bug fix
Signed-off-by: Jarek-Rolski <[email protected]>
* fix pydantic tests for pydantic<2
Signed-off-by: Jarek-Rolski <[email protected]>
* add pytest test_typed_dataframe_model_json_schema
Signed-off-by: Jarek-Rolski <[email protected]>
---------
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Andrej Iring <[email protected]>
* resolve failing pylint tests
Signed-off-by: Andrej Iring <[email protected]>
---------
Signed-off-by: Andrej Iring <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: cosmicBboy <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Jarek-Rolski <[email protected]>format_vectorized_error_message to properly format nested pyarrow failed cases (#2036)1 parent 2e355f7 commit f9b0af5
File tree
2 files changed
+105
-3
lines changed- pandera/backends/pandas
- tests/pandas
2 files changed
+105
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
59 | 60 | | |
60 | 61 | | |
61 | | - | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
513 | 513 | | |
514 | 514 | | |
515 | 515 | | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
0 commit comments