`python::pip`'s `notreallyaversion` is not compatible with latest `pip` (>= 24.1) due to changed output (again)

[kBite]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 8
• Ruby: 3.2
• Distribution: AlmaLinux 9
• Module version: v7.3.0

How to reproduce (e.g Puppet code you use)

python::pip fails to install with ensure => 'latest'

Detecting $latest_version relys on parsing (from versions: ... )

$latest_version = join([
    "${pip_install} ${legacy_resolver} ${pypi_index} ${pypi_extra_index} ${proxy_flag}",
    " ${install_args} ${install_editable} ${real_pkgname}==notreallyaversion 2>&1",
    " | sed -nE 's/.*\\(from versions: (.*, )*(.*)\\)/\\2/p'",
    ' | tr -d "[:space:]"',
])

pip (>= 24.1) does not return (from versions: ... ) anymore.

# pip --version
pip 24.1.2 from /opt/kolla-ansible/venv/lib64/python3.9/site-packages/pip (python 3.9)

# pip install ansible==notreallyaversion
ERROR: Invalid requirement: 'ansible==notreallyaversion': Expected end or semicolon (after name and no valid version specifier)
    ansible==notreallyaversion
           ^

There is an undocumented (https://pip.pypa.io/en/stable/cli/), experimental command, index versions, that returns ...

 # pip index versions ansible
WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
ansible (8.7.0)
Available versions: 8.7.0, 8.6.1, 8.6.0, 8.5.0, 8.4.0, 8.3.0, 8.2.0, 8.1.0, 8.0.0, 7.7.0, 7.6.0, 7.5.0, 7.4.0, 7.3.0, 7.2.0, 7.1.0, 7.0.0, 6.7.0, 6.6.0, 6.5.0, 6.4.0, 6.3.0, 6.2.0, 6.1.0, 6.0.0, 5.10.0, 5.9.0, 5.8.0, 5.7.1, 5.7.0, 5.6.0, 5.5.0, 5.4.0, 5.3.0, 5.2.0, 5.1.0, 5.0.1, 4.10.0, 4.9.0, 4.8.0, 4.7.0, 4.6.0, 4.5.0, 4.4.0, 4.3.0, 4.2.0, 4.1.0, 4.0.0, 3.4.0, 3.3.0, 3.2.0, 3.1.0, 3.0.0, 2.10.7, 2.10.6, 2.10.5, 2.10.4, 2.10.3, 2.10.2, 2.10.1, 2.10.0, 2.9.27, 2.9.26, 2.9.25, 2.9.24, 2.9.23, 2.9.22, 2.9.21, 2.9.20, 2.9.19, 2.9.18, 2.9.17, 2.9.16, 2.9.15, 2.9.14, 2.9.13, 2.9.12, 2.9.11, 2.9.10, 2.9.9, 2.9.8, 2.9.7, 2.9.6, 2.9.5, 2.9.4, 2.9.3, 2.9.2, 2.9.1, 2.9.0, 2.8.20, 2.8.19, 2.8.18, 2.8.17, 2.8.16, 2.8.15, 2.8.14, 2.8.13, 2.8.12, 2.8.11, 2.8.10, 2.8.9, 2.8.8, 2.8.7, 2.8.6, 2.8.5, 2.8.4, 2.8.3, 2.8.2, 2.8.1, 2.8.0, 2.7.18, 2.7.17, 2.7.16, 2.7.15, 2.7.14, 2.7.13, 2.7.12, 2.7.11, 2.7.10, 2.7.9, 2.7.8, 2.7.7, 2.7.6, 2.7.5, 2.7.4, 2.7.3, 2.7.2, 2.7.1, 2.7.0, 2.6.20, 2.6.19, 2.6.18, 2.6.17, 2.6.16, 2.6.15, 2.6.14, 2.6.13, 2.6.12, 2.6.11, 2.6.10, 2.6.9, 2.6.8, 2.6.7, 2.6.6, 2.6.5, 2.6.4, 2.6.3, 2.6.2, 2.6.1, 2.6.0, 2.5.15, 2.5.14, 2.5.13, 2.5.12, 2.5.11, 2.5.10, 2.5.9, 2.5.8, 2.5.7, 2.5.6, 2.5.5, 2.5.4, 2.5.3, 2.5.2, 2.5.1, 2.5.0, 2.4.6.0, 2.4.5.0, 2.4.4.0, 2.4.3.0, 2.4.2.0, 2.4.1.0, 2.4.0.0, 2.3.3.0, 2.3.2.0, 2.3.1.0, 2.3.0.0, 2.2.3.0, 2.2.2.0, 2.2.1.0, 2.2.0.0, 2.1.6.0, 2.1.5.0, 2.1.4.0, 2.1.3.0, 2.1.2.0, 2.1.1.0, 2.1.0.0, 2.0.2.0, 2.0.1.0, 2.0.0.2, 2.0.0.1, 2.0.0.0, 1.9.6, 1.9.5, 1.9.4, 1.9.3, 1.9.2, 1.9.1, 1.9.0.1, 1.8.4, 1.8.3, 1.8.2, 1.8.1, 1.8, 1.7.2, 1.7.1, 1.7, 1.6.10, 1.6.9, 1.6.8, 1.6.7, 1.6.6, 1.6.5, 1.6.4, 1.6.3, 1.6.2, 1.6.1, 1.6, 1.5.5, 1.5.4, 1.5.3, 1.5.2, 1.5.1, 1.5, 1.4.5, 1.4.4, 1.4.3, 1.4.2, 1.4.1, 1.4, 1.3.4, 1.3.3, 1.3.2, 1.3.1, 1.3.0, 1.2.3, 1.2.2, 1.2.1, 1.2, 1.1, 1.0
  INSTALLED: 8.7.0
  LATEST:    8.7.0

... but does not support programatic output yet (pypa/pip#10983).

[kengelhardt-godaddy]

Another option: replace notreallyaversion with 9!0dev0+x

$ pip -V
pip 24.2 from /home/dshaw/.pyenv/versions/3.11.8/lib/python3.11/site-packages/pip (python 3.11)

$ pip install 'requests==9!0dev0+x'
ERROR: Ignored the following yanked versions: 2.32.0, 2.32.1
ERROR: Could not find a version that satisfies the requirement requests==9!0dev0+x (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.11.1, 2.12.0, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.12.5, 2.13.0, 2.14.0, 2.14.1, 2.14.2, 2.15.1, 2.16.0, 2.16.1, 2.16.2, 2.16.3, 2.16.4, 2.16.5, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0, 2.18.1, 2.18.2, 2.18.3, 2.18.4, 2.19.0, 2.19.1, 2.20.0, 2.20.1, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.25.1, 2.26.0, 2.27.0, 2.27.1, 2.28.0, 2.28.1, 2.28.2, 2.29.0, 2.30.0, 2.31.0, 2.32.2, 2.32.3)
ERROR: No matching distribution found for requests==9!0dev0+x

This works because:

• 9! is the epoch, a part of the version specification I've never seen anyone use (please let's keep it that way)
• dev0 refers to being the first possible development version, my understanding is dev versions are not allowed to be uploaded to PyPI
• +x is a local version identifier, my understanding is local identifiers are not allowed to be uploaded to PyPI

• quoted from [24.1] name== hack for listing package versions no longer works pypa/pip#12852 (comment)

[notatallshaw]

I will note, that notreallyaversion was previously a valid legacy version (pre-PEP440), so replacing it with a PEP440 compliant version that is very unlikely to ever be used is no worse, even if it remains non-ideal solution.

[kengelhardt-godaddy]

Totally agree, it's at best a workaround.

An actual solution would leverage the new index versions subcommand once it's stable.