Using csvkit with Python3.9 produces DeprecationWarning's

[saleyn]

When running csvkit tools using Python 3.9, there's a ton of warnings:

$ python --version
Python 3.9.2
$ csvsql ...
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

I tried to disable the warnings with the following, but it doesn't help:

PYTHONWARNINGS="ignore::DeprecationWarning" csvsql

[jpmckinney]

You say a ton of warnings, but only display one. Can you paste more of the warnings?

[saleyn]

They are all identical as shown above. About 1200 per run. It doesn't seem to depend on the number of lines in the CSV file. I am doing:

$ csvsql -i mysql <(echo -en "a,b,c\n1,2,3\n") 2>&1 | wc -l
1234

or

$ csvsql -i mysql <(echo -en "a,b,c\n1,2,3\n") 2>&1 | head -5
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
/usr/lib/python3.9/site-packages/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

[jpmckinney]

This is an upstream issue with setuptools and packaging with respect to Python 3.9:

pypa/packaging#368
pypa/setuptools#2466
pypa/setuptools#2497
pypa/packaging#407

Some projects have implemented a workaround to silence the warning, like qutebrowser/qutebrowser@822bfec

[saleyn]

Can you add a similar workaround to csvkit? Since these are standalone tools, there's nothing much an enduser can do to prevent the warnings.

[saleyn]

An end-user's workaround for this would be using csvsql in this manner:

$ csvsql -i mysql <(echo -en "a,b,c\n1,2,3\n") 2> >(grep -v DeprecationWarning &>/dev/stderr)
CREATE TABLE `63` (
        a BOOL NOT NULL, 
        b DECIMAL(38, 0) NOT NULL, 
        c DECIMAL(38, 0) NOT NULL, 
        CHECK (a IN (0, 1))
);
$ echo $?
0

However, it would be much neater if the proper workaround is implemented within the csvkit.

[jpmckinney]

There's an in-progress pull request by a Pypa maintainer, so I'll wait to a bit to see if that gets closed: pypa/packaging#407

[r0bis]

Sadly this is still a problem in 2023.

/usr/lib/python3.11/site-packages/csvkit/utilities/csvsql.py:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2350: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('zope')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages

This is an amazing program, but warnings are problematic and then if there seems to be an error in response to SQL syntax, checking and rectifying is problematic too.

[jpmckinney]

@r0bis what version of csvkit are you using?

There is nothing at the reported line using pkg_resources

[jpmckinney]

The upstream issues I linked to have been closed, so the original issue is closed.

@r0bis If you believe there is an issue on the most recent version of csvkit, file an issue following the instructions with all necessary details to reproduce. https://github.com/wireservice/csvkit/blob/master/CONTRIBUTING.rst

[r0bis]

Thanks, I submitted the issue #1215 - version is 1.1.1 and it is manjaro linux