Skip to content

Conversation

@desertaxle
Copy link
Member

@desertaxle desertaxle commented Dec 23, 2025

The Docket worker cancellation is causing the server to hang on shutdown. This PR doesn't solve the underlying issue, but it does prevent the server from hanging by adding a max time that the server will wait for Docket worker shutdown.

Closes #19924

Related to chrisguidry/docket#260

@github-actions github-actions bot added the bug Something isn't working label Dec 23, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Dec 23, 2025

CodSpeed Performance Report

Merging #19940 will not alter performance

Comparing fix-server-shutdown-hang (2850cbe) with main (985ef41)

Summary

✅ 2 untouched

@desertaxle desertaxle merged commit 79265ab into main Dec 23, 2025
61 of 63 checks passed
@desertaxle desertaxle deleted the fix-server-shutdown-hang branch December 23, 2025 16:47
Comment on lines +69 to +75
logger.debug(
"Waiting for background worker to finish after cancellation..."
)
await asyncio.wait_for(worker_task, timeout=5.0)
logger.debug(
"Background worker finished successfully after cancellation"
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@desertaxle I'm working on a fix for this in Docket which should be safer because it will still allow any currently running tasks to finish. I'll let you know as soon as the fix is released

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@desertaxle The fix is in Docket 0.16.2 if you want to give it a test without needing the timeout here.

I'm also noticing that this function __aexit__s the Worker context while the worker.run_forever() task is still going, that might cause some issues with the Worker's internal state variables. It would be better to have the async with Worker(...): cover the worker task cancellation and cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ctrl + C ability to terminate a prefect server start running in foreground

4 participants