Running tests inside namespace packages ?

[asmodehn]

I am having import issues with namespaces packages and pytest.
Something along the lines of pytest already having the namespace imported and not looking deeper into the subpackage for the actual package (while running the AssertionRewritingHook:find_modules method in rewrite.py)

I am currently trying to reproduce the issue on a minimal example, so I made https://github.com/asmodehn/python_ns_pkgs.

But it seems the test on these installed packages are not even collected...

Envrionment setup :

alexv@AlexV-Linux:~$ git clone https://github.com/asmodehn/python_ns_pkgs
Cloning into 'python_ns_pkgs'...
remote: Counting objects: 25, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 25 (delta 3), reused 25 (delta 3), pack-reused 0
Unpacking objects: 100% (25/25), done.
Checking connectivity... done.

alexv@AlexV-Linux:~$ mkvirtualenv pytest_nspkgs
New python executable in /home/alexv/.virtualenvs/pytest_nspkgs/bin/python
Installing setuptools, pip, wheel...done.

(pytest_nspkgs) alexv@AlexV-Linux:~$ cd python_ns_pkgs/
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ tree
.
├── foo1
│   ├── foo
│   │   ├── foo
│   │   │   ├── __init__.py
│   │   │   └── test_import.py
│   │   └── __init__.py
│   └── setup.py
└── foo2
    ├── foo
    │   ├── bar
    │   │   ├── __init__.py
    │   │   └── test_import.py
    │   └── __init__.py
    └── setup.py
6 directories, 8 files
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pip install foo1/
Processing ./foo1
Installing collected packages: foo1
  Running setup.py install for foo1 ... done
Successfully installed foo1-1.0
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pip install foo2/
Processing ./foo2
Installing collected packages: foo2
  Running setup.py install for foo2 ... done
Successfully installed foo2-1.0
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ ls ~/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo
bar  foo
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ tree ~/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo
/home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo
├── bar
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── test_import.py
│   └── test_import.pyc
└── foo
    ├── __init__.py
    ├── __init__.pyc
    ├── test_import.py
    └── test_import.pyc

2 directories, 8 files
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ python -c "from foo import foo, bar"
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ 

The things I attempted with pytest so far :

(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo.bar
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo.bar (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo.foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo.foo (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest ~/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/foo/test_import.py
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv, inifile:
plugins: hypothesis-3.1.0
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
_______________________________________________________________________________________ ERROR collecting .virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/foo/test_import.py ________________________________________________________________________________________
ImportError while importing test module '/home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/foo/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/foo/test_import.py:1: in <module>
    from foo import foo,bar
E   ImportError: cannot import name foo
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.11 seconds ===========================================================================================================================
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest ~/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/bar/test_import.py
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv, inifile:
plugins: hypothesis-3.1.0
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
_______________________________________________________________________________________ ERROR collecting .virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/bar/test_import.py ________________________________________________________________________________________
ImportError while importing test module '/home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/bar/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages/foo/bar/test_import.py:1: in <module>
    from foo import foo,bar
E   ImportError: cannot import name foo
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.06 seconds ===========================================================================================================================
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ cd foo1/
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ ls foo/
foo  __init__.py
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ python -c "import foo.bar"
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ python -c "import broken"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named broken
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ pytest foo/foo/test_import.py 
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs/foo1, inifile:
plugins: hypothesis-3.1.0
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
___________________________________________________________________________________________________________________ ERROR collecting foo/foo/test_import.py ___________________________________________________________________________________________________________________
ImportError while importing test module '/home/alexv/python_ns_pkgs/foo1/foo/foo/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
foo/foo/test_import.py:1: in <module>
    from foo import foo,bar
E   ImportError: cannot import name bar
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.06 seconds ===========================================================================================================================
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ python -c "from foo import foo, bar"
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ 

then I realize I might have been using an old pytest version, so I ran these :

(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ pip install pytest
Collecting pytest
/home/alexv/.virtualenvs/pytest_nspkgs/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/alexv/.virtualenvs/pytest_nspkgs/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Using cached pytest-3.0.7-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages (from pytest)
Collecting py>=1.4.29 (from pytest)
  Using cached py-1.4.33-py2.py3-none-any.whl
Requirement already satisfied: appdirs>=1.4.0 in /home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages (from setuptools->pytest)
Requirement already satisfied: packaging>=16.8 in /home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages (from setuptools->pytest)
Requirement already satisfied: six>=1.6.0 in /home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages (from setuptools->pytest)
Requirement already satisfied: pyparsing in /home/alexv/.virtualenvs/pytest_nspkgs/lib/python2.7/site-packages (from packaging>=16.8->setuptools->pytest)
Installing collected packages: py, pytest
Successfully installed py-1.4.33 pytest-3.0.7
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ which pytest
/home/alexv/.virtualenvs/pytest_nspkgs/bin/pytest
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ pytest --pyargs foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs/foo1, inifile:
plugins: hypothesis-3.1.0
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
___________________________________________________________________________________________________________________ ERROR collecting foo/foo/test_import.py ___________________________________________________________________________________________________________________
ImportError while importing test module '/home/alexv/python_ns_pkgs/foo1/foo/foo/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
foo/foo/test_import.py:1: in <module>
    from foo import foo,bar
E   ImportError: cannot import name bar
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.06 seconds ===========================================================================================================================
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs/foo1$ cd ..
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo.bar
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo.bar (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ pytest --pyargs foo.foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/python_ns_pkgs, inifile:
plugins: hypothesis-3.1.0

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo.foo (missing __init__.py?)
(pytest_nspkgs) alexv@AlexV-Linux:~/python_ns_pkgs$ 

But still no success...

So

• How can I run these test modules ?
• Why the packages foo.foo and foo.bar are not found even though python finds them ?
• Does pytest support running tests from installed namespace packages at all ?

Thanks to shed some light on these...

[RonnyPfannschmidt]

at first glance that is a standard python level error,

please run commands like: python -c 'import foo.bar.test_packages' to try and replicate out of pytest

[asmodehn]

I setup a new clean environment :

alexv@alexv-pc:~/Projects$ git clone https://github.com/asmodehn/python_ns_pkgs
Cloning into 'python_ns_pkgs'...
remote: Counting objects: 25, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 25 (delta 3), reused 25 (delta 3), pack-reused 0
Unpacking objects: 100% (25/25), done.
Checking connectivity... done.
alexv@alexv-pc:~/Projects$ cd python_ns_pkgs/
alexv@alexv-pc:~/Projects/python_ns_pkgs$ mkvirtualenv ns_pkgs_test
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/alexv/.virtualenvs/ns_pkgs_test/bin/python2
Also creating executable in /home/alexv/.virtualenvs/ns_pkgs_test/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

I first did an install with python setup.py install which didn't work apparently :

(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ cd foo1
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs/foo1$ python setup.py install
running install
running bdist_egg
running egg_info
creating foo1.egg-info
writing foo1.egg-info/PKG-INFO
writing namespace_packages to foo1.egg-info/namespace_packages.txt
writing top-level names to foo1.egg-info/top_level.txt
writing dependency_links to foo1.egg-info/dependency_links.txt
writing manifest file 'foo1.egg-info/SOURCES.txt'
writing manifest file 'foo1.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/foo
creating build/lib.linux-x86_64-2.7/foo/foo
copying foo/foo/test_import.py -> build/lib.linux-x86_64-2.7/foo/foo
copying foo/foo/__init__.py -> build/lib.linux-x86_64-2.7/foo/foo
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/foo
creating build/bdist.linux-x86_64/egg/foo/foo
copying build/lib.linux-x86_64-2.7/foo/foo/test_import.py -> build/bdist.linux-x86_64/egg/foo/foo
copying build/lib.linux-x86_64-2.7/foo/foo/__init__.py -> build/bdist.linux-x86_64/egg/foo/foo
byte-compiling build/bdist.linux-x86_64/egg/foo/foo/test_import.py to test_import.pyc
byte-compiling build/bdist.linux-x86_64/egg/foo/foo/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying foo1.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo1.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo1.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo1.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo1.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/foo1-1.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing foo1-1.0-py2.7.egg
Copying foo1-1.0-py2.7.egg to /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages
Adding foo1 1.0 to easy-install.pth file

Installed /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo1-1.0-py2.7.egg
Processing dependencies for foo1==1.0
Finished processing dependencies for foo1==1.0
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs/foo1$ cd ../foo2
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs/foo2$ python setup.py install
running install
running bdist_egg
running egg_info
creating foo2.egg-info
writing foo2.egg-info/PKG-INFO
writing namespace_packages to foo2.egg-info/namespace_packages.txt
writing top-level names to foo2.egg-info/top_level.txt
writing dependency_links to foo2.egg-info/dependency_links.txt
writing manifest file 'foo2.egg-info/SOURCES.txt'
writing manifest file 'foo2.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/foo
creating build/lib.linux-x86_64-2.7/foo/bar
copying foo/bar/test_import.py -> build/lib.linux-x86_64-2.7/foo/bar
copying foo/bar/__init__.py -> build/lib.linux-x86_64-2.7/foo/bar
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/foo
creating build/bdist.linux-x86_64/egg/foo/bar
copying build/lib.linux-x86_64-2.7/foo/bar/test_import.py -> build/bdist.linux-x86_64/egg/foo/bar
copying build/lib.linux-x86_64-2.7/foo/bar/__init__.py -> build/bdist.linux-x86_64/egg/foo/bar
byte-compiling build/bdist.linux-x86_64/egg/foo/bar/test_import.py to test_import.pyc
byte-compiling build/bdist.linux-x86_64/egg/foo/bar/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying foo2.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo2.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo2.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo2.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying foo2.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/foo2-1.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing foo2-1.0-py2.7.egg
Copying foo2-1.0-py2.7.egg to /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages
Adding foo2 1.0 to easy-install.pth file

Installed /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo2-1.0-py2.7.egg
Processing dependencies for foo2==1.0
Finished processing dependencies for foo2==1.0
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs/foo2$ cd ..
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ python -c 'import foo.bar.test_import'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named foo.bar.test_import
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ ls ~/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/
appdirs-1.4.3.dist-info/       easy-install.pth               foo1-1.0-py2.7.egg             packaging-16.8.dist-info/      pkg_resources/                 pyparsing.py                   setuptools-35.0.0.dist-info/   six.pyc
appdirs.py                     easy_install.py                foo2-1.0-py2.7.egg             pip/                           pkg_resources-0.0.0.dist-info/ pyparsing.pyc                  six-1.10.0.dist-info/          wheel/
appdirs.pyc                    easy_install.pyc               packaging/                     pip-9.0.1.dist-info/           pyparsing-2.2.0.dist-info/     setuptools/                    six.py                         wheel-0.30.0a0.dist-info/

Not exactly sure why, it might be one of these bugs : https://github.com/jonparrott/namespace-pkg-tests/blob/master/table.md

However I removed and reinstall with pip :

(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pip uninstall foo1 foo2
Uninstalling foo1-1.0:
  /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo1-1.0-py2.7.egg
Proceed (y/n)? y
  Successfully uninstalled foo1-1.0
Uninstalling foo2-1.0:
  /home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo2-1.0-py2.7.egg
Proceed (y/n)? y
  Successfully uninstalled foo2-1.0
Traceback (most recent call last):
  File "/home/alexv/.virtualenvs/ns_pkgs_test/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/__init__.py", line 233, in main
    return command.main(cmd_args)
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/basecommand.py", line 252, in main
    pip_version_check(session)
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/utils/outdated.py", line 102, in pip_version_check
    installed_version = get_installed_version("pip")
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 838, in get_installed_version
    working_set = pkg_resources.WorkingSet()
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 644, in __init__
    self.add_entry(entry)
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 700, in add_entry
    for dist in find_distributions(entry, True):
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1949, in find_eggs_in_zip
    if metadata.has_metadata('PKG-INFO'):
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1463, in has_metadata
    return self.egg_info and self._has(self._fn(self.egg_info, name))
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1823, in _has
    return zip_path in self.zipinfo or zip_path in self._index()
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1703, in zipinfo
    return self._zip_manifests.load(self.loader.archive)
  File "/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1643, in load
    mtime = os.stat(path).st_mtime
OSError: [Errno 2] No such file or directory: '/home/alexv/.virtualenvs/ns_pkgs_test/local/lib/python2.7/site-packages/foo1-1.0-py2.7.egg'
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pip uninstall foo1 foo2
Cannot uninstall requirement foo1, not installed
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pip uninstall foo2
Cannot uninstall requirement foo2, not installed
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pip install foo1/
Processing ./foo1
Installing collected packages: foo1
  Running setup.py install for foo1 ... done
Successfully installed foo1-1.0
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pip install foo2/
Processing ./foo2
Installing collected packages: foo2
  Running setup.py install for foo2 ... done
Successfully installed foo2-1.0

And got it to work, for python, but not for pytest :

(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ ls ~/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/
appdirs-1.4.3.dist-info  easy-install.pth  foo                       foo2-1.0-py2.7.egg-info   packaging-16.8.dist-info  pkg_resources                  pyparsing.py   setuptools-35.0.0.dist-info  six.pyc
appdirs.py               easy_install.py   foo1-1.0-py2.7.egg-info   foo2-1.0-py2.7-nspkg.pth  pip                       pkg_resources-0.0.0.dist-info  pyparsing.pyc  six-1.10.0.dist-info         wheel
appdirs.pyc              easy_install.pyc  foo1-1.0-py2.7-nspkg.pth  packaging                 pip-9.0.1.dist-info       pyparsing-2.2.0.dist-info      setuptools     six.py                       wheel-0.30.0a0.dist-info

(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ python -c 'from foo import foo, bar'
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ python -c 'import foo.foo.test_import'
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pytest --pyargs foo.foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/Projects/python_ns_pkgs, inifile:
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
______________________________________________________________________________________________________________________________ ERROR collecting  ______________________________________________________________________________________________________________________________
ImportError while importing test module '/home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo/foo/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ImportError: No module named test_import
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.06 seconds ===========================================================================================================================
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pytest --pyargs foo.foo.test_import
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/Projects/python_ns_pkgs, inifile:
collected 0 items / 1 errors 

=================================================================================================================================== ERRORS ====================================================================================================================================
______________________________________________________________________________________________________________________________ ERROR collecting  ______________________________________________________________________________________________________________________________
ImportError while importing test module '/home/alexv/.virtualenvs/ns_pkgs_test/lib/python2.7/site-packages/foo/foo/test_import.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ImportError: No module named test_import
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================================================================== 1 error in 0.06 seconds ===========================================================================================================================
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pytest --pyargs foo
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/Projects/python_ns_pkgs, inifile:

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
ERROR: file or package not found: foo (missing __init__.py?)
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pytest --pyargs foo.bar
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/Projects/python_ns_pkgs, inifile:
collected 0 items 

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ pytest --pyargs foo.bar.test_import
============================================================================================================================= test session starts =============================================================================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/alexv/Projects/python_ns_pkgs, inifile:
collected 0 items 

======================================================================================================================== no tests ran in 0.00 seconds =========================================================================================================================
(ns_pkgs_test) alexv@alexv-pc:~/Projects/python_ns_pkgs$ python -c 'import foo.bar.test_import'

It seems :

• only foo.foo.test_import is found, not foo.bar.test_import. (a bug I didn't expect)
• foo.foo.test_import cannot be imported by pytest. (the bug I originally intended to show here)

[ssbarnea]

@RonnyPfannschmidt This should be moved to discussions.

[nicoddemus]

Was about to do it but noticed the link is not there anymore. Seems like they have removed that feature: community/community#2952 (comment) 😞 (see the more recent replies)

[woutdenolf]

Since pytest only supports legacy namespace packages I'd like to bump this issue once again.

Soon pkg-resources will disappear so we are replacing all our pkg-resources style namespace packages with native namespace packages. pytest does not support them as described in this issue.

A minimal working example can be found here: https://github.com/woutdenolf/pytest_pep420.

pytest --import-mode=importlib solves the issue but has important limitations: test modules are non-importable by each other, utility modules in the tests directories are not automatically importable.

[bluetech]

@woutdenolf Did you try https://docs.pytest.org/en/stable/reference/reference.html#confval-consider_namespace_packages?

[woutdenolf]

Thanks @bluetech ! Adding this to pyproject.toml solves my issue

[tool.pytest.ini_options]
consider_namespace_packages = true

Run

git clone https://github.com/woutdenolf/pytest_pep420.git
cd pytest_pep420
pip install -e .[test]
pytest -x

Result

(pybox_9S5crM) :~/tmp/pytest_pep420$ pytest -x
======================================== test session starts =========================================
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/denolf/tmp/pytest_pep420
configfile: pyproject.toml
collected 2 items                                                                                    

src/mynamespace/myproject/tests/test_myproject_ns.py .                                         [ 50%]
src/myproject/tests/test_myproject.py .                                                        [100%]

========================================= 2 passed in 0.01s ==========================================