Support Groups as Extras

[jaymegordo]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

There seems to be a few places where this issue is mentioned, but I don't see anything explicitly tracking it.

(Specifically @matthijskooijman's comment (point 2) 1644#issuecomment-895848818)

The new groups functionality is very helpful when managing packages installed while developing a project, but seems to miss a pretty good use case - basically using groups in place of extras when installing this package in another environment.

Instead of having to do:

# myproject_1

[tool.poetry.dependencies]
package_1 = "^0.0.1"

[tool.poetry.group.group_a]
optional = true

# this lets someone else develop this project with optional deps
[tool.poetry.group.group_a.dependencies]
package_2 = {version = "^0.0.2", optional = true}
package_3 = {version = "^0.0.3", optional = true}

# need to explicitly define "extras" for other packages install optional deps
[tool.poetry.extras]
group_a = ["package_2", "package_3"]

It would be cleaner (and make more sense I think, once groups is out of preview), if we could just do:

# myproject_1

[tool.poetry.dependencies]
package_1 = "^0.0.1"

[tool.poetry.group.group_a]
optional = true

[tool.poetry.group.group_a.dependencies]
package_2 = "^0.0.2"
package_3 = "^0.0.3"

# didn't need to explicitly define "extras", or individual packages as "optional"

And then in myproject_2:

# myproject_2

[tool.poetry.dependencies]
myproject_1 = {version = "^1.0.0", extras = ["group_a"]}

[jaymegordo]

Side note (not sure if this should be a separate issue), but it would also be helpful if poetry export supported groups as well.

My use case for this is that part of my code base is run as an azure functions app which has to be built with pip/requirements.txt. It would be helpful if I could do poetry export --with group_a,group_b.

[Conchylicultor]

From #4891,

Currently, using group & extras together will silently fail (package don't get installed but no error is raised)

[wk8]

+1, this would be awesome to have.

[AdamWRichardson]

A lot of other comments seem to say that groups and extras are different things and extras are for runtime dependencies and groups are for development dependencies. This is a fair distinction except some tools (such as tox) are well used in development and does separate installation of the package. A well known work around is to use the extras essentially as groups.

In short a big +1 for this feature as it allows grouping in toml but still outputting PEP compliant extras

[neersighted]

We have an FAQ entry about using tox -- note patterns 2 and 3 work with the use of groups, so what you want may already be possible in this case.

I agree, this seems like a huge missed opportunity, groups are much nicer to work with. Perhaps if it's deemed necessary groups could be annotated with category = "main" / category = "dev" (or dev = false / dev = true)

[neersighted]

You're describing optional and non-optional groups -- non-optional (default) groups label and sort your dependencies but are always in the tree. Optional groups are used as a replacement for dev dependencies.

Extras are inherently different as they can include the same package multiple time -- the current design of groups is not designed with similar functionality in mind. Given extras are a regular feature of the Python ecosystem, the current design for them is focused on familiarity and parity with the rest of the ecosystem.

It is certainly possible to add interactions between groups and extras, but no one on the core team is (to the best of my knowledge) yet thinking about or working on such a feature. A cohesive design proposal/MR would be welcome, but would likely have to go through a decent amount of iteration to get it mergable.

Also, any work on changing the project format is likely on hold until we can stabilize PEP 621 support.

[jacek-jablonski]

How can we elegantly solve this problem?

[tool.poetry.dependencies]
python = "~3.10"
asyncpg = { version = "*", optional = true }

[tool.poetry.group.dev.dependencies]
pytest = "^7.2.0"
pytest-asyncio = "^0.20.2"
pytest-cov = "^4.0.0"
pytest-mock = "^3.10.0"

asyncpg-stubs = { version = "*", optional = true }

[tool.poetry.extras]
asyncpg = ["asyncpg", "asyncpg-stubs"]

Command poetry install --all-extras --with dev doesn't install asyncpg-stubs.