codeautolink_custom_blocks breaks Sphinx's caching mechanism

[mgeier]

Issue

When using codeautolink_custom_blocks, which is supposed to be a dictionary containing a callable, Sphinx will re-build all HTML files every time it is running, even if no source files have changed.

Expected behavior

No re-building of HTML files if nothing changes.

Steps to reproduce

Add this to conf.py:

codeautolink_custom_blocks = {
    'python': lambda x: (x, x),
}

Run Sphinx.
Run Sphinx again. Notice that the HTML files are re-built even though nothing changed

Possible solutions

The current behavior should be kept for backwards compatibility, but probably in addition to a callable, a string could also be supported?

This could probably be implemented using sphinx.util.import_object() (for an example see https://github.com/spatialaudio/nbsphinx/blob/44829f13d18e676917fc6e7b7ae62d4bac625cd7/src/nbsphinx.py#L990-L991).

For a bit more background (I was struggling with this some time ago), see this mailing list thread: https://groups.google.com/g/sphinx-dev/c/aGfJb-TC8PI/m/zxM3l9kAAgAJ

[felix-hilden]

I think this is not likely a bug. Although the caching was something I hadn't considered, I think we still need the callable API for functions that are not importable. But your suggestion about adding a string interface certainly works too. So is your goal here to implement another extension that depends on codeautolink and would provide the parser function? Otherwise, if a user is writing (often just wrapping) the parser, they would use the configuration file.

And to be clear we're still deep in beta, so if we deem the callable API worthless we're 100% removing it 😄

[mgeier]

I think we still need the callable API for functions that are not importable.

Sure, I was suggesting to keep it.

So is your goal here to implement another extension that depends on codeautolink and would provide the parser function?

Not really.
But maybe someone else will.

Otherwise, if a user is writing (often just wrapping) the parser, they would use the configuration file.

Sure, if they are willing to disable caching.

But if caching is important to them, they can simply create a new Python file in the source directory and put their custom block function in there. Then it can be specified as a string and imported with sphinx.util.import_object().

[felix-hilden]

Yeah let's do it! I was just trying to see where you came from 👍 Thanks for verifying that the approach works!