bugfix/1927 #2019
Merged
bugfix/1927 #2019
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Jarek-Rolski <[email protected]>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2019 +/- ##
===========================================
- Coverage 94.28% 80.29% -13.99%
===========================================
Files 91 121 +30
Lines 7013 9754 +2741
===========================================
+ Hits 6612 7832 +1220
- Misses 401 1922 +1521 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
Signed-off-by: Jarek-Rolski <[email protected]>
cosmicBboy
approved these changes
Jun 26, 2025
AndrejIring
pushed a commit
to AndrejIring/pandera
that referenced
this pull request
Jun 30, 2025
* fix generating pydantic core schema for generic types Signed-off-by: Jarek-Rolski <[email protected]> * test pytest for typed generic DataFrame Signed-off-by: Jarek-Rolski <[email protected]> * fix issue with docs gitactions test Signed-off-by: Jarek-Rolski <[email protected]> * fix pytest test_typed_generic_dataframe Signed-off-by: Jarek-Rolski <[email protected]> * fix pytest test_typed_generic_dataframe Signed-off-by: Jarek-Rolski <[email protected]> * improve changes Signed-off-by: Jarek-Rolski <[email protected]> * bug fix Signed-off-by: Jarek-Rolski <[email protected]> * fix pydantic tests for pydantic<2 Signed-off-by: Jarek-Rolski <[email protected]> * add pytest test_typed_dataframe_model_json_schema Signed-off-by: Jarek-Rolski <[email protected]> --------- Signed-off-by: Jarek-Rolski <[email protected]> Signed-off-by: Andrej Iring <[email protected]>
cosmicBboy
added a commit
that referenced
this pull request
Jul 2, 2025
…ed pyarrow failed cases (#2036) * resolve 2035 Signed-off-by: Andrej Iring <[email protected]> * change astype to apply Signed-off-by: Andrej Iring <[email protected]> * bugfix: PANDERA_VALIDATION_ENABLED=False should disable validation (#2028) * bugfix: PANDERA_VALIDATION_ENABLED=False should disable validation Signed-off-by: Niels Bantilan <[email protected]> * fix tests Signed-off-by: Niels Bantilan <[email protected]> --------- Signed-off-by: Niels Bantilan <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Expect Python slice index errors after Python 3.10 (#2033) * Expect Python slice index errors with pandas again Seen in https://github.com/unionai-oss/pandera/actions/runs/15526426636/job/43706977542?pr=2030 Signed-off-by: Deepyaman Datta <[email protected]> * Expect Python slice index errors after Python 3.10 Signed-off-by: Deepyaman Datta <[email protected]> * Fix the version check to not include Python 3.10.x Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * resolve failing pyspark tests Signed-off-by: Andrej Iring <[email protected]> * add new test Signed-off-by: Andrej Iring <[email protected]> * format code using black Signed-off-by: Andrej Iring <[email protected]> * Add `DataFrameModel`, `DataFrameSchema` for `ibis` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Refactor Ibis `DataFrameSchema` to extend pandas's Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add code for basic `Column`, and stub more modules Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add Ibis's parsing, validation, and error backends Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement stub to validate schema component checks Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement validation for floating types Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Inherit getter, only override setter, for `.dtype` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * implement column, add basic unit tests for Ibis data types Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add `ibis` extra and regenerate requirements files Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Re-enable Python equivalents for `int` and `float` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Do not test Ibis below py3.9, add DuckDB for tests (#1773) * Restore accidentally-deleted use of "breakpoint()" Signed-off-by: Deepyaman Datta <[email protected]> * Swap `types-pkg_resources` with `types-setuptools` (#1779) * Swap `types-pkg_resources` with `types-setuptools` Signed-off-by: Deepyaman Datta <[email protected]> * Update the expected error message to name variable Signed-off-by: Deepyaman Datta <[email protected]> * Fix expected outputs for pandas-stubs 2.2.2.240807 Signed-off-by: Deepyaman Datta <[email protected]> * Update CI configuration to filter right pandas ver Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> * Do not test Ibis below py3.9, add DuckDB for tests Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Support specifying and validating optional columns (#1762) * Test model schema equivalency with optional dtypes Signed-off-by: Deepyaman Datta <[email protected]> * Support specifying and validating optional columns Signed-off-by: Deepyaman Datta <[email protected]> * Add DuckDB for testing and regenerate requirements Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Register string datatype and update existing tests (#1766) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add missing `type` field for the `Int32` data type (#1771) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement minimal `coerce` and corresponding tests (#1772) * Implement minimal `coerce` and corresponding tests Signed-off-by: Deepyaman Datta <[email protected]> * Do not exclude 3.9 in Ibis CI, and regenerate reqs Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Ibis check backend (#1831) * [wip] add minimal ibis check backend implementation Signed-off-by: cosmicBboy <[email protected]> * support scalar, column, and table check output types Signed-off-by: cosmicBboy <[email protected]> * support scalar, column, and table check output types Signed-off-by: cosmicBboy <[email protected]> * Ibis check backend suggestions (#1855) * Apply suggestions from code review Signed-off-by: Deepyaman Datta <[email protected]> * Fix row-order-dependent order by adding table cols Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> * fix lint Signed-off-by: cosmicBboy <[email protected]> * fix unit tests Signed-off-by: cosmicBboy <[email protected]> --------- Signed-off-by: cosmicBboy <[email protected]> Signed-off-by: Deepyaman Datta <[email protected]> Co-authored-by: Deepyaman Datta <[email protected]> Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Remove Ibis backend's use of `scalar_failure_case` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Update signature of `run_schema_components_checks` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Fix violations in `pandera/backends//container.py` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Remove use of multimethod library and align Polars Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * fix backend registration Signed-off-by: Niels Bantilan <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement minimal built-in checks for Ibis backend (#1885) * Implement minimal built-in checks for Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> * Implement `Column` validation for the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> * Promote check object to table during preprocessing Signed-off-by: Deepyaman Datta <[email protected]> * Remove extraneous fixture for backend registration Signed-off-by: Deepyaman Datta <[email protected]> * Resolve lint (unused imports, undefined variables) Signed-off-by: Deepyaman Datta <[email protected]> * Partially standardize docstrings of builtin checks Signed-off-by: Deepyaman Datta <[email protected]> * Fix the `preprocess` docstrings copied from pandas Signed-off-by: Deepyaman Datta <[email protected]> * Format pandera/backends/ibis/checks.py using Black Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Update test to reflect scalar output since Ibis 10 (#1907) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Use `Union` type for `PysparkObject` to fix typing Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add Ibis with the DuckDB extra as a dev dependency Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Update/align noxfile.py and generated requirements Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Supress unused argument check for `data_container` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Include `polars` extra dep for testing Ibis dtypes Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Update check example to correctly process IbisData Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Execute validated schema to get the desired result Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement "ne" built-in check for the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement int/uint/float types, except for float16 Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement timestamp type, and test built-in checks Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Support built-in checks for interval-typed columns Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Blacken pandera/engines/ibis_engine.py module code Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `dt.Date` type, and test built-in checks Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `dt.Time` type, and test built-in checks Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `gt` and `ge` check for the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Standardize docstrings, don't say "data container" Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `lt` and `le` check for the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Fix "form" to "from", and align docstring for test Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Apply suggestion from Copilot to fix error message Co-authored-by: Copilot <[email protected]> Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Support table-level checks, including for built-in Signed-off-by: Deepyaman Datta <[email protected]> Co-authored-by: cosmicBboy <[email protected]> Signed-off-by: cosmicBboy <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `is_in_range` check for the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `isin()` built-in check for Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Refactor duplicated code into `_across()` function Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `notin()` builtin check for Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement the `str_matches` built-in check on Ibis Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `str_contains` check on the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `str_startswith()` check on Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `str_endswith()` built-in check for Ibis Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement the `str_length` built-in check for Ibis Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement the unique_values_eq built-in Ibis check Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add entry for `ibis` to the backend support matrix Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Write detailed documentation on using Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Replace `ibis.expr.types` imports (where possible) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Implement `typing` module and decorators for Ibis Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Duplicate tests for Ibis decorators implementation Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Use `code-cell` over `testcode` to build Ibis docs Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Add Ibis docs to the supported libraries `toctree` Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Create element-wise checks automatically for users (#2043) * Create element-wise checks automatically for users Signed-off-by: Deepyaman Datta <[email protected]> * Mark unsupported scenario with element-wise checks Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Remove `to_format` methods that don't work on Ibis (#2044) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Execute dataframe-level checks on the Ibis backend (#2041) * Execute dataframe-level checks on the Ibis backend Signed-off-by: Deepyaman Datta <[email protected]> * Remove unnecessary, un-Pythonic array length check Signed-off-by: Deepyaman Datta <[email protected]> * Remove bit on separately running data-level checks Signed-off-by: Deepyaman Datta <[email protected]> --------- Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * Mark unsupported scenario with element-wise checks (#2046) Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * handle dataframe-level failure cases: convert row to dict Signed-off-by: cosmicBboy <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * bugfix/1927 (#2019) * fix generating pydantic core schema for generic types Signed-off-by: Jarek-Rolski <[email protected]> * test pytest for typed generic DataFrame Signed-off-by: Jarek-Rolski <[email protected]> * fix issue with docs gitactions test Signed-off-by: Jarek-Rolski <[email protected]> * fix pytest test_typed_generic_dataframe Signed-off-by: Jarek-Rolski <[email protected]> * fix pytest test_typed_generic_dataframe Signed-off-by: Jarek-Rolski <[email protected]> * improve changes Signed-off-by: Jarek-Rolski <[email protected]> * bug fix Signed-off-by: Jarek-Rolski <[email protected]> * fix pydantic tests for pydantic<2 Signed-off-by: Jarek-Rolski <[email protected]> * add pytest test_typed_dataframe_model_json_schema Signed-off-by: Jarek-Rolski <[email protected]> --------- Signed-off-by: Jarek-Rolski <[email protected]> Signed-off-by: Andrej Iring <[email protected]> * resolve failing pylint tests Signed-off-by: Andrej Iring <[email protected]> --------- Signed-off-by: Andrej Iring <[email protected]> Signed-off-by: Niels Bantilan <[email protected]> Signed-off-by: Deepyaman Datta <[email protected]> Signed-off-by: cosmicBboy <[email protected]> Signed-off-by: Jarek-Rolski <[email protected]> Co-authored-by: Niels Bantilan <[email protected]> Co-authored-by: Deepyaman Datta <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Jarek-Rolski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #1927 Deltares/Ribasim#2120
In
__get_pydantic_core_schema__function I added condition to check ifpydantic-corepackage version is above2.30.0orDataFrameunderlying type isTypeVar, when it is positive the function executescore_schema.no_info_plain_validator_functionwithoutjson_schema_input_schemaandserializastionparameters.I also added a few tests checking if
pydanticmodels are created correctly.