Skip to content

test: refactor integration suite and switch mixed fixtures to cffi#3068

Merged
messense merged 11 commits into
PyO3:mainfrom
messense:refactor-tests
Mar 8, 2026
Merged

test: refactor integration suite and switch mixed fixtures to cffi#3068
messense merged 11 commits into
PyO3:mainfrom
messense:refactor-tests

Conversation

@messense

@messense messense commented Mar 7, 2026

Copy link
Copy Markdown
Member

Summary

  • refactor the integration test suite into behavior-oriented modules and shared case-driven helpers
  • remove cargo-mock and make mutable cffi mixed fixtures run from isolated per-case package copies
  • switch most mixed develop/integration fixtures from pyo3 to cffi, update copied fixture docs, and remove unused pyo3-only mixed fixtures

Testing

  • cargo test --test run -- --list
  • cargo nextest run

@messense messense linked an issue Mar 7, 2026 that may be closed by this pull request
@messense messense requested a review from Copilot March 7, 2026 11:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR restructures the Rust integration test suite into multiple behavior-focused modules, removes the old cargo-mock acceleration mechanism, and updates several “mixed” fixtures to use CFFI (including per-case isolated fixture copies for mutable/generated artifacts).

Changes:

  • Split tests/run.rs into multiple modules under tests/run/ (develop, integration, sdist, wheel, pep517, environment, errors).
  • Introduced case-driven helpers in tests/common/ for per-case target/wheel directories, environment preparation, and isolated fixture copying (test-crates/case-packages/<case_id>).
  • Replaced several pyo3 “mixed” fixtures with equivalent CFFI fixtures; removed test-crates/cargo-mock and updated contributing docs accordingly.

Reviewed changes

Copilot reviewed 85 out of 104 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/run.rs Turns the test crate into a module dispatcher for the new run suite.
tests/run/develop.rs Adds parameterized develop tests using DevelopCase.
tests/run/environment.rs Extracts environment-related regressions (SOURCE_DATE_EPOCH, musl, unreadable dir).
tests/run/errors.rs Extracts error-path regressions into a dedicated module.
tests/run/integration.rs Adds parameterized integration tests using IntegrationCase and fixture copy pruning.
tests/run/pep517.rs Adds PEP517 profile tests via Pep517Case.
tests/run/sdist.rs Extracts and refactors sdist regressions with case helpers and imperative preserved cases.
tests/run/wheel.rs Extracts wheel content/path assertions (incl. optional SBOM).
tests/common/mod.rs Adds shared test helpers: per-case dirs, env prep, fixture copying, prereq install.
tests/common/develop.rs Refactors develop runner to accept DevelopCase + shared helpers.
tests/common/integration.rs Refactors integration runner to accept IntegrationCase + shared helpers + isolated fixture copies.
tests/common/pep517.rs Refactors PEP517 runner to accept Pep517Case + shared env/dir helpers.
tests/common/other.rs Improves recursive copy skipping artifacts; factors out wheel_files; uses per-case dirs.
.gitignore Ignores test-crates/case-packages/ created by per-case fixture isolation.
guide/src/contributing.md Removes cargo-mock instructions; keeps faster-tests guidance.
test-dockerfile.sh Reduces the set of crates tested in the docker helper script.
test-crates/cargo-mock/src/main.rs Removes cargo-mock implementation.
test-crates/cargo-mock/Cargo.toml Removes cargo-mock crate manifest.
test-crates/cargo-mock/Cargo.lock Removes cargo-mock lockfile.
test-crates/pyo3-mixed-submodule/tox.ini Removes obsolete pyo3 mixed-submodule fixture test config.
test-crates/pyo3-mixed-submodule/tests/test_pyo3_mixed_submodule.py Removes obsolete pyo3 mixed-submodule fixture test.
test-crates/pyo3-mixed-submodule/src/lib.rs Removes obsolete pyo3 mixed-submodule Rust code.
test-crates/pyo3-mixed-submodule/pyo3_mixed_submodule/init.py Removes obsolete pyo3 mixed-submodule Python package init.
test-crates/pyo3-mixed-submodule/check_installed/check_installed.py Removes obsolete pyo3 mixed-submodule smoke script.
test-crates/pyo3-mixed-submodule/README.md Removes obsolete pyo3 mixed-submodule README.
test-crates/pyo3-mixed-submodule/Cargo.toml Removes obsolete pyo3 mixed-submodule manifest.
test-crates/pyo3-mixed-submodule/Cargo.lock Removes obsolete pyo3 mixed-submodule lockfile.
test-crates/pyo3-mixed-implicit/tox.ini Removes obsolete pyo3 mixed-implicit fixture test config.
test-crates/pyo3-mixed-implicit/tests/test_pyo3_mixed_implicit.py Removes obsolete pyo3 mixed-implicit fixture test.
test-crates/pyo3-mixed-implicit/src/lib.rs Removes obsolete pyo3 mixed-implicit Rust code.
test-crates/pyo3-mixed-implicit/python/pyo3_mixed_implicit/some_rust/init.py Removes obsolete pyo3 mixed-implicit Python init.
test-crates/pyo3-mixed-implicit/check_installed/check_installed.py Removes obsolete pyo3 mixed-implicit smoke script.
test-crates/pyo3-mixed-implicit/README.md Removes obsolete pyo3 mixed-implicit README.
test-crates/pyo3-mixed-implicit/Cargo.toml Removes obsolete pyo3 mixed-implicit manifest.
test-crates/pyo3-mixed-implicit/Cargo.lock Removes obsolete pyo3 mixed-implicit lockfile.
test-crates/pyo3-mixed-with-path-dep/tox.ini Removes obsolete pyo3 mixed-with-path-dep fixture test config.
test-crates/pyo3-mixed-with-path-dep/tests/test_pyo3_mixed_with_path_dep.py Removes obsolete pyo3 mixed-with-path-dep fixture test.
test-crates/pyo3-mixed-with-path-dep/src/lib.rs Removes obsolete pyo3 mixed-with-path-dep Rust code.
test-crates/pyo3-mixed-with-path-dep/pyo3_mixed_with_path_dep/init.py Removes obsolete pyo3 mixed-with-path-dep Python init.
test-crates/pyo3-mixed-with-path-dep/check_installed/check_installed.py Removes obsolete pyo3 mixed-with-path-dep smoke script.
test-crates/pyo3-mixed-with-path-dep/README.md Removes obsolete pyo3 mixed-with-path-dep README.
test-crates/pyo3-mixed-with-path-dep/Cargo.lock Removes obsolete pyo3 mixed-with-path-dep lockfile.
test-crates/pyo3-mixed-with-path-dep/.gitignore Removes obsolete pyo3 mixed-with-path-dep ignore file.
test-crates/cffi-mixed-with-path-dep/src/lib.rs Adds CFFI FFI surface for the “with path dep” fixture.
test-crates/cffi-mixed-with-path-dep/pyproject.toml Renames project + adds dependencies=["cffi"] and bindings="cffi".
test-crates/cffi-mixed-with-path-dep/check_installed/check_installed.py Adds CFFI-based install smoke test.
test-crates/cffi-mixed-with-path-dep/cffi_mixed_with_path_dep/init.py Exposes ffi/lib for the fixture.
test-crates/cffi-mixed-with-path-dep/README.md Documents new CFFI fixture usage.
test-crates/cffi-mixed-with-path-dep/Cargo.toml Renames crate/lib from pyo3 to CFFI variant.
test-crates/cffi-mixed-with-path-dep/Cargo.lock Adds lockfile for the new fixture.
test-crates/cffi-mixed-with-path-dep/.gitignore Adds ignore rules for Python artifacts.
test-crates/cffi-mixed-submodule/src/lib.rs Adds CFFI-exported FFI functions/types for submodule fixture.
test-crates/cffi-mixed-submodule/pyproject.toml Switches to CFFI bindings and updates module name.
test-crates/cffi-mixed-submodule/check_installed/check_installed.py Adds smoke test importing generated submodule.
test-crates/cffi-mixed-submodule/cffi_mixed_submodule/rust_module/init.py Adds Python package init for rust submodule.
test-crates/cffi-mixed-submodule/cffi_mixed_submodule/python_module/double.py Adds small Python helper module.
test-crates/cffi-mixed-submodule/cffi_mixed_submodule/python_module/init.py Adds (empty) package init.
test-crates/cffi-mixed-submodule/cffi_mixed_submodule/init.py Exposes rust submodule import.
test-crates/cffi-mixed-submodule/README.md Documents new CFFI fixture usage.
test-crates/cffi-mixed-submodule/Cargo.toml Adds new crate manifest for CFFI submodule fixture.
test-crates/cffi-mixed-submodule/Cargo.lock Adds lockfile for the new fixture.
test-crates/cffi-mixed-submodule/.gitignore Adds ignore rules (currently appears mismatched).
test-crates/cffi-mixed-src/src/cffi_mixed_src/python_module/double.py Adds Python helper module for src-layout fixture.
test-crates/cffi-mixed-src/src/cffi_mixed_src/python_module/init.py Adds (empty) package init.
test-crates/cffi-mixed-src/src/cffi_mixed_src/init.py Exposes ffi/lib for the src-layout fixture.
test-crates/cffi-mixed-src/rust/src/lib.rs Adds CFFI FFI surface for src-layout fixture.
test-crates/cffi-mixed-src/rust/Cargo.toml Adds Rust crate manifest for src-layout fixture.
test-crates/cffi-mixed-src/rust/Cargo.lock Adds lockfile for src-layout fixture.
test-crates/cffi-mixed-src/pyproject.toml Adds pyproject for CFFI src-layout fixture.
test-crates/cffi-mixed-src/check_installed/check_installed.py Adds smoke test for src-layout fixture.
test-crates/cffi-mixed-src/README.md Documents new CFFI src-layout fixture.
test-crates/cffi-mixed-src/.gitignore Updates ignores (adds target/).
test-crates/cffi-mixed-py-subdir/src/lib.rs Adds CFFI FFI surface for python-subdir fixture.
test-crates/cffi-mixed-py-subdir/python/cffi_mixed_py_subdir/python_module/double.py Adds Python helper module.
test-crates/cffi-mixed-py-subdir/python/cffi_mixed_py_subdir/python_module/init.py Adds (empty) package init.
test-crates/cffi-mixed-py-subdir/python/cffi_mixed_py_subdir/init.py Exposes generated module via ffi/lib.
test-crates/cffi-mixed-py-subdir/pyproject.toml Adds pyproject defining python-source/include/module-name for CFFI.
test-crates/cffi-mixed-py-subdir/check_installed/check_installed.py Adds smoke test for python-subdir fixture.
test-crates/cffi-mixed-py-subdir/assets/extra_data.txt Adds included data file for include-pattern regression.
test-crates/cffi-mixed-py-subdir/README.md Documents new CFFI python-subdir fixture.
test-crates/cffi-mixed-py-subdir/Cargo.toml Adds Rust crate manifest for python-subdir fixture.
test-crates/cffi-mixed-py-subdir/Cargo.lock Adds lockfile for python-subdir fixture.
test-crates/cffi-mixed-py-subdir/.gitignore Adds ignore rules (currently appears mismatched).
test-crates/cffi-mixed-include-exclude/src/lib.rs Adds CFFI FFI surface for include/exclude regression fixture.
test-crates/cffi-mixed-include-exclude/pyproject.toml Adds include/exclude rules + out-dir include mapping for regression coverage.
test-crates/cffi-mixed-include-exclude/check_installed/check_installed.py Adds smoke test verifying out-dir included file content.
test-crates/cffi-mixed-include-exclude/cffi_mixed_include_exclude/python_module/double.py Adds Python helper module.
test-crates/cffi-mixed-include-exclude/cffi_mixed_include_exclude/python_module/init.py Adds (empty) package init.
test-crates/cffi-mixed-include-exclude/cffi_mixed_include_exclude/exclude_this_file Adds excluded file target for exclude rule.
test-crates/cffi-mixed-include-exclude/cffi_mixed_include_exclude/init.py Exposes generated module via ffi/lib.
test-crates/cffi-mixed-include-exclude/cffi_mixed_include_exclude/.gitignore Ignores included/generated files within python package.
test-crates/cffi-mixed-include-exclude/build.rs Generates generated_info.txt into OUT_DIR for include mapping.
test-crates/cffi-mixed-include-exclude/README.md Documents new include/exclude CFFI fixture.
test-crates/cffi-mixed-include-exclude/Cargo.toml Adds Rust crate manifest for include/exclude fixture.
test-crates/cffi-mixed-include-exclude/Cargo.lock Adds lockfile for include/exclude fixture.
test-crates/cffi-mixed-include-exclude/.gitignore Adds ignore rules (currently appears mismatched).
test-crates/cffi-mixed-implicit/src/lib.rs Adds CFFI FFI surface for implicit-namespace fixture.
test-crates/cffi-mixed-implicit/python/cffi_mixed_implicit/some_rust/init.py Exposes generated rust submodule via ffi/lib.
test-crates/cffi-mixed-implicit/pyproject.toml Switches to CFFI bindings + updates module-name/python-source.
test-crates/cffi-mixed-implicit/check_installed/check_installed.py Adds smoke test for implicit-namespace fixture.
test-crates/cffi-mixed-implicit/README.md Documents new implicit-namespace CFFI fixture.
test-crates/cffi-mixed-implicit/Cargo.toml Adds Rust crate manifest for implicit fixture.
test-crates/cffi-mixed-implicit/Cargo.lock Adds lockfile for implicit fixture.
test-crates/cffi-mixed-implicit/.gitignore Adds ignore rules (currently appears mismatched).

Comment thread tests/common/integration.rs Outdated
Comment thread test-crates/cffi-mixed-submodule/.gitignore Outdated
Comment thread test-crates/cffi-mixed-include-exclude/.gitignore Outdated
Comment thread test-crates/cffi-mixed-implicit/.gitignore Outdated
Comment thread test-crates/cffi-mixed-py-subdir/.gitignore Outdated
Comment thread tests/run/develop.rs Outdated
@messense messense requested a review from Copilot March 7, 2026 12:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 88 out of 106 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

tests/common/integration.rs:1

  • The subprocess wrapper sets CARGO_BIN_EXE_cargo-zigbuild to the maturin test binary path (CARGO_BIN_EXE_maturin). That’s very likely incorrect and can cause zig builds to invoke the wrong executable. Set this env var to the actual cargo-zigbuild test binary (e.g., env!(\"CARGO_BIN_EXE_cargo-zigbuild\")) or remove this override if it’s not needed for the subprocess path.
    tests/common/mod.rs:1
  • fs::remove_dir_all / fs::remove_file are used here, but fs isn’t imported in the shown module header (only env and str were kept in the diff). This will fail to compile unless std::fs is already imported elsewhere in the file. Fix by adding use std::fs; (or fully-qualifying these calls with std::fs::...).
    tests/run/integration.rs:1
  • This test is named integration_pyo3_pure_conda / id integration-pyo3-pure-conda, but it runs the test-crates/pyo3-mixed fixture. This mismatch makes failures hard to interpret. Either switch the fixture path to test-crates/pyo3-pure or rename the test/id to ...pyo3-mixed-conda to match what’s being exercised.

messense added 4 commits March 7, 2026 20:44
…tants, and deduplication

- Add constructor methods to DevelopCase, IntegrationCase, Pep517Case to
  eliminate repeated default fields in rstest case definitions
- Extract 6 shared TestPackageCopy constants to deduplicate fixture copy
  configs between develop and integration test modules
- Extract install_and_check_wheel helper in integration.rs to deduplicate
  pip install + error checking between zig and non-zig paths
- Rename PreparedEnv.root to env_dir for clarity with both venv and conda
- Harden fixture_root_for_package with explicit strip_prefix validation
- Remove dead manylinux tag assertion after zig early return
- Remove unused PlatformTag import
@messense messense merged commit fdab97f into PyO3:main Mar 8, 2026
44 of 45 checks passed
@messense messense deleted the refactor-tests branch March 8, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch most integration tests to cffi

2 participants