Skip to content

Conversation

sshane
Copy link
Contributor

@sshane sshane commented Jul 26, 2025

each worker pytest -nx is set up sequentially, and takes about 0.04 to 0.05s to spawn a new Python instance . If you run pytest -n32 on a file, ~2s is just startup. this doesn't have a huge impact in CI because there's only 8 workers (~0.4s)

the line that takes the most time per node/worker:

https://github.com/pytest-dev/pytest-xdist/blob/239def9be9a489526f0a56e6ba442149f17cb919/src/xdist/workermanage.py#L106

pytest-xdist PR: pytest-dev/pytest-xdist#1229


When you run a file with 1 test with -n0 vs -n1, there is still a difference of >0.05s, so xdist still is doing other operations taking up time that should be able to be optimized, but I don't know how much we want to spend on this. pytest does seem to be the best runner (tooling, test API, ease of use) from what I've seen looking around online.

@sshane sshane changed the title faster xdist startup pytest: faster xdist startup Jul 26, 2025
@sshane sshane merged commit e461d18 into master Jul 26, 2025
18 checks passed
@sshane sshane deleted the faster-xdist branch July 26, 2025 03:29
@maxime-desroches
Copy link
Collaborator

You also need to update the lock file with uv lock

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants