Drain held events once connected - #8698
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8698 +/- ##
========================================
Coverage 86.13% 86.14%
========================================
Files 348 348
Lines 57171 57287 +116
========================================
+ Hits 49247 49350 +103
- Misses 7924 7937 +13 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
philippjfr
commented
Aug 4, 2026
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8692.
Background
When a
hold()block exits while the Document is not yet connected, thefinallychain inpanel/io/document.pytook:This clears the hold policy but leaves
doc.callbacks._held_eventspopulated. Bokeh'sunhold()returns early when_hold is None, so nothing drains them there. The events sit on the Document until some later, unrelatedhold()/unhold()pair flushes them.The branch is deliberate. It arrived in #7972 to avoid dispatching events before the Document is fully initialized, which leads to the pending writes error. The intent was that these events be discarded rather than deferred: model changes occurring during startup don't need to be synced, because they are picked up anyway when the whole Document is serialized on connect.
The problem
That premise only holds if the change actually reaches the model, and pre-connect it often doesn't.
state._unblocked(doc)isFalsebefore the session connects, soReactive._apply_updatetakes its else branch and schedules the update viastate.execute(cb, schedule=True), which isdoc.add_next_tick_callback. That emits aSessionCallbackAdded, which is not aDocumentPatchedEventand so is neither dispatched nor recoverable from a full serialization. Stranding it means the callback is never registered on the IOLoop and the deferred_update_modelnever runs.So the single
SessionCallbackAddedin the issue's reproducer is the pane update. Instrumenting that reproducer to also read the model:This is not late or out-of-context dispatch. The server-side model is left stale, and the full serialization on connect then faithfully sends the stale state. The same applies to anything else that registers through
SessionCallbackAdded:add_next_tick_callback,add_periodic_callback,state.execute(schedule=True)and the scheduled branch ofReactive._send_event.Two further gaps in the same chain:
The
threadedbranch never consulted_connectedat all. It unconditionally scheduleddoc.unhold()on a next tick, so the discard policy was simply not implemented there, and the dispatch it did instead is the pending-writes hazard #7972 was avoiding, reached by a different door. Withconfig.nthreadsset this is a common path, since_schedule_on_loadsubmits_on_loadto the thread pool.The non-threaded branch mutated
_holdwithout taking_HOLD_LOCK[doc], so it raced the threaded unhold.The fix
Resolve the events at hold exit rather than parking them, sorting into three groups instead of discarding wholesale:
SessionCallbackAdded/SessionCallbackRemoved. These are how callbacks get registered on the IOLoop; dropping them loses the work they represent. Dispatching pre-connect is safe because the Document lock is held at this point.MessageSentEvent. It is aDocumentPatchedEvent, but it carries protocol messages (custom events fromReactive._send_event, ipywidgets comm messages) with no representation in the model graph, so a full serialization cannot reproduce them. They cannot be dispatched at hold exit either, because there are no subscribed connections yet to write them to, so they are held in a_UNCONNECTED_EVENTSweak-keyed map and flushed from_on_loadonce_connectedis set. The flush goes through a next-tick callback so it lands on the Document's thread with the lock held, since_on_loadmay run on the thread pool.ModelChangedEvent,TitleChangedEvent,RootAdded/RootRemovedEventand theColumnDataSourcestream/patch hints are all applied to the model before the event is emitted, so the serialization on connect reproduces them. Worth noting for the CDS case:PropertyValueColumnData._streamand._patchdeliberately bypass the wrapped dict methods and write through to the underlying data, so.datais locally correct and the events are transport hints rather than the state itself.The threaded branch now applies the same policy, so both agree on what happens pre-connect, and the drain takes
_HOLD_LOCK[doc]. Events are collected under the lock but dispatched after releasing it, since the lock is not reentrant and a change callback may itself enterhold()._UNCONNECTED_EVENTSis cleared in_destroy_documentalongside the other per-Document maps.Testing
Four tests in
panel/tests/io/test_document.py:test_hold_before_connected_does_not_strand_events— nothing is left queued, and the scheduled callback runs so the model update lands.test_hold_before_connected_defers_message_sent—MessageSentEventis neither dropped nor stranded.test_hold_before_connected_drops_recoverable_events— a model change is dropped, with the value present on the model.test_threaded_hold_before_connected_does_not_strand_events— the threaded branch leaves no hold in place while unconnected. Asserts it actually took the threaded path.The first and last were confirmed to fail against the pre-fix code (
queued == ['SessionCallbackAdded']andhold_value == 'combine'respectively), so they pin the regression rather than just passing.panel/tests/io/,panel/tests/test_server.pyandpanel/tests/test_reactive.pyshow no regressions. Threetest_server_ico_handling*failures and thetest_resources/test_reactivestylesheet failures are present onmainin this working tree and unrelated to this change.🤖 Generated with Claude Code