[Docs] Clarification on Package Discovery wording "exactly to the directory structure"

[kennethjmyers]

Summary

In the Packaged Discovery page (docs link, github link), the very first example on the use of packages in the [options] could benefit from a project directory example(s) like almost every other example on that page. The part that I'm looking for clarification on is this part (emphasis mine):

If your packages are not in the root of the repository or do not correspond exactly to the directory structure, you also need to configure package_dir:

What is meant by "exactly to the directory structure"? Does this mean you can specify the package is the root level name and if so then I think it should just say that rather than calling it the "directory structure"? Furthermore what happens if there is a folder in the root level that is the same name as the root level and that name is used such as in the following example:

mypkg
├── pyproject.toml  # AND/OR setup.cfg, setup.py
├── ...
└── mypkg/
    └── __init__.py

setup.cfg:

[options]
#...
packages =
    mypkg

OS / Environment

N/A

Additional Information

The wording on this has just been confusing to me and I feel it could confuse others. I have an internal company package where the creator is no longer here (and unfortunately I can't share) that used this method of package discovery to seemingly define the root level as the package and I did not understand how they did that or why that worked based on the documentation text.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[abravalheri]

Hi @kennethjmyers,

In the Packaged Discovery page (docs link, github link), the very first example on the use of packages in the [options] could benefit from a project directory example(s) like almost every other example on that page.

We are looking for volunteers to help update the documentations. Please feel free to open PRs if you feel you can help.

What is meant by "exactly to the directory structure"? Does this mean you can specify the package is the root level name and if so then I think it should just say that rather than calling it the "directory structure"?

Let me try to clarify this by example. Consider you have the following "directory structure" in your repository:

root_project_directory/
├── pyproject.toml
├── a/
│   └── b/
│       └── c.py
└── d/
    └── e.py

But you want to create a package that when installed will provide the following files:

mypkg/
├── c.py
└── d/ 
    └── e.py

Note here how the files stored in the project repository do not correspond to the same directory structure as the package author wants to be installed at the end user's machine.

Furthermore what happens if there is a folder in the root level that is the same name as the root level and that name is used such as in the following example:

This example should be OK. The files should be installed at the end-user's machine as:

mypkg
└── __init__.py

This kind directory structure would however be problematic in editable installations if the package is a "namespace package" rather than a "regular package" (in the context of implicit namespace packages as defined in PEP 420).

Personally, I recommend the src-layout as the least error prone way of organising the source code for a Python package (there is a discussion of benefits here: https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/).

[kennethjmyers]

Hey @abravalheri thank you for the quick and thorough reply on this. I think I understand now. When you have a setup.cfg file like the one in the docs example:

[options]
#...
packages =
    mypkg
    mypkg.subpkg1
    mypkg.subpkg2

mypkg is just the name of the top level import, but that doesn't necessarily correspond to an actual folder in the directory structure, is that correct? If so then I think the documentation could benefit from something added about that because it wasn't clear to me that the first package translated that way. If I'm understanding that correctly I will try to suggest a modification to it in a PR.

I agree with what you said about the src-layout, that has been my go to choice when working on projects in the past.

[abravalheri]

Yeah, users can customise that by "remapping" package_dir (although that is not advisable).