Fix JSON serialization error in Tabulator with mixed NaT datetime columns

[SuMayaBee]

Description

Fixes a bug where pn.widgets.Tabulator crashes with ValueError: Out of range float values are not JSON compliant when a datetime column contains a mix of pd.NaT and valid values after calling .dt.date.

While investigating, I found that the problem wasn’t in Bokeh’s NaN handling itself but in how Panel registers a custom encoder for pd.NaT. Panel was calling Serializer.register(pd.NaT, ...), but Bokeh’s Serializer._encode() looks up encoders by type(obj), so type(pd.NaT) (NaTType) never matched that registration. As a result, pd.NaT fell through to the default datetime path (convert_datetime_type), which returned a raw float('nan'). That raw NaN then ended up in the payload and caused serialize_json to raise the JSON error.

Problem:
Mixed pd.NaT/datetime columns ended up sending raw float('nan') through Bokeh’s JSON encoder because the custom encoder for pd.NaT was registered on the instance instead of its type.

Solution:
Register the encoder on type(pd.NaT) so Bokeh’s Serializer can actually find and apply it:

from bokeh.core.serialization import Serializer

import pandas as pd

# in Tabulator._process_df_and_convert_to_cds
try:
    Serializer.register(type(pd.NaT), lambda _, __: None)
except AssertionError:
    pass

This is effectively a one‑word change (pd.NaT → type(pd.NaT)) and keeps the rest of the Tabulator code unchanged. No per‑column or per‑row scanning is needed anymore.

Example:

import pandas as pd
import panel as pn

df = pd.DataFrame({"date": [pd.NaT, 12]})
df["date"] = pd.to_datetime(df["date"]).dt.date
tabulator = pn.widgets.Tabulator(df)
tabulator.show()  # ✅ Now works correctly (previously raised the JSON ValueError)

Fixes #7803

How Has This Been Tested?

Added a regression test that exercises the same code path as the original failure:

Locally verified:

• With the old Serializer.register(pd.NaT, ...) code, this test raises the ValueError.
• With Serializer.register(type(pd.NaT), ...), the same test passes.

Also re-checked that:

• All‑NaT, no‑NaT, and mixed NaT/non‑NaT datetime columns still render correctly in Tabulator.

AI Disclosure

• This PR contains AI-generated content.
  • I have tested all AI-generated content in my PR.
  • I take responsibility for all AI-generated content in my PR.
    Tools: Gemini 3 Flash – Used to analyze the bug, identify the root cause, and refine the implementation and tests.

Checklist

• Tests added and is passing
• Added documentation

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.06%. Comparing base (b74f3b1) to head (b23f377).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8491      +/-   ##
==========================================
- Coverage   86.18%   86.06%   -0.12%     
==========================================
  Files         349      349              
  Lines       55048    55058      +10     
==========================================
- Hits        47444    47388      -56     
- Misses       7604     7670      +66     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hoxbro]

This seems like a lot of computation for an edge case.

[mattpap]

json.encoder hits float('nan') → ValueError: Out of range float values are not JSON compliant

By that time any NaNs should have been converted to an internal, JSON-compatible representation by Bokeh, e.g.:

In [1]: from bokeh.core.serialization import Serializer

In [2]: s = Serializer()

In [3]: s.encode(float("nan"))
Out[3]: {'type': 'number', 'value': 'nan'}

NaNs in ndarrays and similar are also handled. So either this analysis is wrong or there's a bug somewhere in Bokeh.

[SuMayaBee]

This seems like a lot of computation for an edge case.

@hoxbro Removed the heavy computation. The fix is now just correcting the Serializer registration key, which is O(1) and happens once per Tabulator instance.

[hoxbro]

That looks much better. Can you add a test. You should verify that the tests fails before and works after the fix.

[SuMayaBee]

json.encoder hits float('nan') → ValueError: Out of range float values are not JSON compliant

By that time any NaNs should have been converted to an internal, JSON-compatible representation by Bokeh, e.g.:

In [1]: from bokeh.core.serialization import Serializer

In [2]: s = Serializer()

In [3]: s.encode(float("nan"))
Out[3]: {'type': 'number', 'value': 'nan'}

NaNs in ndarrays and similar are also handled. So either this analysis is wrong or there's a bug somewhere in Bokeh.

@mattpap Good catch, you’re right that Bokeh’s Serializer already knows how to handle NaNs. The problem turned out to be on the Panel side: it was calling Serializer.register(pd.NaT, ...), but Serializer._encode() looks up encoders by type(obj), so type(pd.NaT) (NaTType) never hit that registration and pd.NaT went through convert_datetime_type() to a raw float('nan'), which then blew up JSON encoding. I’ve switched this to Serializer.register(type(pd.NaT), ...), so it uses the correct key and the extra per-column workaround is no longer needed.

[philippjfr]

@mattpap Actually could you clarify

By that time any NaNs should have been converted to an internal, JSON-compatible representation by Bokeh, e.g.:

That was my thinking too but it seems like ColumnDataSource.data does not get handled in this way, e.g. here's the serialized version of the example above:

'source': {'type': 'object', 'name': 'ColumnDataSource', 'id': '47d529de-92ea-4351-8632-70b9381aaa8b', 'attributes': {'selected': {'type': 'object', 'name': 'Selection', 'id': '53138583-7626-4106-bc0e-369e761f57a7', 'attributes': {'indices': [], 'line_indices': []}}, 'selection_policy': {'type': 'object', 'name': 'UnionRenderers', 'id': 'a65991e5-2f63-45fd-a6f4-46aa970a0d39'}, 'data': {'type': 'map', 'entries': [('index', {'type': 'ndarray', 'array': {'type': 'bytes', 'data': 'H4sIAAEAAAAC/2NgYGBgBGIADLie3QgAAAA='}, 'shape': [2], 'dtype': 'int32', 'order': 'little'}), ('date', {'type': 'ndarray', 'array': [nan, '1970-01-01'], 'shape': [2], 'dtype': 'object', 'order': 'little'})]}}}, 'cell_styles': {'type': 'map', 'entries': [('id', '4210f66b80d345628e29a36eee9a2d22'), ('data', {'type': 'map'})]}

Guessing the internals of array data is untouched by the serializer.

[mattpap]

Guessing the internals of array data is untouched by the serializer.
{'type': 'ndarray', 'array': [nan, '1970-01-01'], ...

This is clearly a bug or some misguided attempt at performance optimization (a bug anyway). This should have been encoded the same way as any list would have been.

[SuMayaBee]

That looks much better. Can you add a test. You should verify that the tests fails before and works after the fix.

@hoxbro @philippjfr I’ve added a regression test for this. In the screen recording I first reverted back to the old code and the test raised the ValueError, then I reapplied the fix and the test passed. So the fix is working and the issue is resolved now.

2026-03-09.14-42-20.mp4

[hoxbro]

LGTM, but can you update the first post with the new findings?

[SuMayaBee]

LGTM, but can you update the first post with the new findings?

@hoxbro Done, updated the PR description. ✅

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.