-
Notifications
You must be signed in to change notification settings - Fork 2k
fix: properly drain workers in prefect_test_harness when used in async contexts #19770
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
zzstoatzz
merged 3 commits into
main
from
devin/1765652244-fix-test-harness-async-cleanup
Dec 17, 2025
Merged
fix: properly drain workers in prefect_test_harness when used in async contexts #19770
zzstoatzz
merged 3 commits into
main
from
devin/1765652244-fix-test-harness-async-cleanup
Dec 17, 2025
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…c contexts Fixes #19762 When prefect_test_harness is used in an async context, the drain_all() and drain() methods return coroutines that were never awaited, causing RuntimeWarning: coroutine 'wait' was never awaited. This fix uses the underlying _drain() method which returns a concurrent.futures.Future that can be waited on synchronously, regardless of whether there's a running event loop. This avoids creating unawaited coroutines in the first place. Changes: - Use _drain() instead of drain()/drain_all() in prefect_test_harness cleanup - Wait synchronously on the futures with concurrent.futures.wait() - Copy EventsWorker instances before iterating to avoid dictionary mutation error - Add regression test that fails if unawaited coroutines are created Co-Authored-By: Nate Nowack <[email protected]>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
CodSpeed Performance ReportMerging #19770 will not alter performanceComparing Summary
|
desertaxle
requested changes
Dec 15, 2025
…lic API usage Address reviewer feedback to avoid using private _drain() methods. Instead, use a wrapper coroutine that calls drain()/drain_all() and awaits the result if it's awaitable. This ensures the awaitable is created and awaited on the same loop, avoiding cross-loop issues. Also move gc import to top of test file per project conventions. Co-Authored-By: Nate Nowack <[email protected]>
zzstoatzz
reviewed
Dec 17, 2025
zzstoatzz
approved these changes
Dec 17, 2025
desertaxle
approved these changes
Dec 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #19762
When
prefect_test_harnessis used in an async context, thedrain_all()anddrain()methods return coroutines that are never awaited, causingRuntimeWarning: coroutine 'wait' was never awaited.Root cause: The
drain()anddrain_all()methods in_QueueServiceBaseare "dual-mode" - they return awaitables when called from an async context (whenget_running_loop()is not None). Sinceprefect_test_harnessis a sync context manager, these returned coroutines were being dropped without being awaited.Solution: Use a wrapper coroutine passed to
run_coro_as_syncthat calls the publicdrain()anddrain_all()methods and awaits them if they return awaitables. This ensures the awaitable is created and awaited on the same loop, avoiding cross-loop issues while using public APIs.Key changes:
drain_workers()async helper that callsAPILogWorker.instance().drain()andEventsWorker.drain_all()inspect.isawaitable()and await them if sorun_coro_as_sync()to handle both sync and async contextsPytestUnraisableExceptionWarningHuman review checklist:
APILogWorkerhasn't been startedChecklist
<link to issue>"mint.json.Link to Devin run: https://app.devin.ai/sessions/70901d7018934b0f9269d6e42042e4a3
Requested by: Nate Nowack ([email protected]) (@zzstoatzz)