Skip to content

fix(testing): Prevent logging setup when --help is passed #2923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 1, 2025

Conversation

esafak
Copy link
Contributor

@esafak esafak commented Aug 1, 2025

This commit fixes a flaky test, test_zipapp_help, which would fail intermittently due to unexpected output on stderr.

The root cause of the flakiness was that virtualenv would set up its logging framework based on verbosity flags (-v, -vv, etc.) before processing the --help flag. This could lead to log messages being generated during the initialization process, which, under certain conditions, would be written to stderr, causing the test to fail.

The fix is to check for the presence of -h or --help in the command-line arguments within the _do_report_setup function. If either of these flags is found, the logging setup is skipped entirely. This is safe because the application will exit immediately after printing the help message, so no logging is required.

This change ensures that no logging output is generated when you request help, which resolves the flaky test and makes the application's behavior more predictable.

Following the fix, the @pytest.mark.flaky annotation has been removed from the test_zipapp_help test, as it is no longer needed.

I am not sure how to name the news fragment when there is no corresponding Issue so I copied an example in commit history.

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

This commit fixes a flaky test, `test_zipapp_help`, which would
fail intermittently due to unexpected output on stderr.

The root cause of the flakiness was that `virtualenv` would set up
its logging framework based on verbosity flags (`-v`, `-vv`, etc.)
*before* processing the `--help` flag. This could lead to log
messages being generated during the initialization process, which,
under certain conditions, would be written to stderr, causing the
test to fail.

The fix is to check for the presence of `-h` or `--help` in the
command-line arguments within the `_do_report_setup` function. If
either of these flags is found, the logging setup is skipped entirely.
This is safe because the application will exit immediately after
printing the help message, so no logging is required.

This change ensures that no logging output is generated when you
request help, which resolves the flaky test and makes the
application's behavior more predictable.

Following the fix, the `@pytest.mark.flaky` annotation has been
removed from the `test_zipapp_help` test, as it is no longer needed.

Signed-off-by: Emre Şafak <[email protected]>
@gaborbernat gaborbernat merged commit ec1c83e into pypa:main Aug 1, 2025
42 checks passed
@esafak esafak deleted the fix/flaky-zipapp-help-test branch August 1, 2025 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants