Skip to content

Conversation

romaingd
Copy link
Contributor

@romaingd romaingd commented Mar 11, 2025

Description of changes

Hi! This PR proposes dropping pytz from primary dependencies and replacing it with stdlib equivalents, as recommended by pytz docs:

Projects using Python 3.9 or later should be using the support now included as part of the standard library, and third party packages work with it such as tzdata. pytz offers no advantages beyond backwards compatibility with code written for earlier versions of Python.

Given that Ibis only supports Python 3.9 and higher, this should be pretty safe.

@github-actions github-actions bot added tests Issues or PRs related to tests impala The Apache Impala backend dependencies Issues or PRs related to dependencies bigquery The BigQuery backend datatypes Issues relating to ibis's datatypes (under `ibis.expr.datatypes`) flink Issues or PRs related to Flink labels Mar 11, 2025
Comment on lines +141 to +142
# adding apache-flink would lock pyarrow to an old version (for all backends),
# due to the transitional dependency apache-beam imposing a low upper bound.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not linked to pytz, but while testing changes in Flink backend I wondered why apache-flink was never installed, and it took me some time to find the explanation in #8696 (comment). I figured I might as well add a comment to save future readers some time. I can split it into a separate PR if deemed more appropriate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Yes, this is definitely a bit hairy at the moment and will be until the Flink team can support a later version of Arrow.

"sqlglot>=23.4",
"toolz>=0.11",
"typing-extensions>=4.3.0",
"tzdata>=2022.7", # fallback time zone data on Windows
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From tzdata documentation on PyPI:

This is a Python package containing zic-compiled binaries for the IANA time zone database. It is intended to be a fallback for systems that do not have system time zone data installed (or don’t have it installed in a standard location), as a part of PEP 615

Unfortunately, without it Windows tests all fail due to a lack of TZ information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference, pandas uses "tzdata>=2022.7" as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safe to only install this on Windows (using a platform selector like sys_platform == 'win32')?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be sufficient to cover the OS we test against (Ubuntu & MacOS have system TZ data, Windows would have the fallback), but I don't know if less popular OS / other Linux distributions come with system time zone data, so it's a bit of a gamble - one that I personally wouldn't take.

@romaingd romaingd marked this pull request as ready for review March 11, 2025 22:04
Copy link
Member

@cpcloud cpcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small question about tzdata, but not blocking. Thanks for doing this!

@cpcloud cpcloud merged commit 3ecf731 into ibis-project:main Mar 12, 2025
110 checks passed
@cpcloud cpcloud added the refactor Issues or PRs related to refactoring the codebase label Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery The BigQuery backend datatypes Issues relating to ibis's datatypes (under `ibis.expr.datatypes`) dependencies Issues or PRs related to dependencies flink Issues or PRs related to Flink impala The Apache Impala backend refactor Issues or PRs related to refactoring the codebase tests Issues or PRs related to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants