Inline code span opening on one line and closing on the next isn't matched, and breaks highlighting for a subsequent header

[denysshnurenko]

What happened

An inline code span (single backtick) whose opening backtick is on one line
and closing backtick is on the next line (a valid CommonMark pattern — a
code span can contain a line ending, normalized to a space) isn't recognized
as a matched pair. After that point, markdown highlighting for the rest of
the document desyncs — specifically, the next ##/### header no longer
gets its header color, rendering as plain text instead.

Minimal repro:

- **Shape**: `SomeType { fieldA: X|Y, fieldB:
  Z, fieldC?: Int }`; more prose here.

## Next Section

Expected: "## Next Section" renders in the normal header color.
Actual: it renders as plain body text — the header styling is lost.

Expected

• A code span split across two lines by a soft wrap or a genuine line
  break within the same paragraph should still be recognized as matched.
• Even if a particular span is misdetected, that should stay contained to
  itself — it should not desync highlighting for markdown elements
  (headers, etc.) later in the document.

Environment

• revdiff version: v1.4.1-5b631a9-20260522T171942
• macOS

[umputun]

had a look, the underlying bug is real but the minimal repro as written doesn't actually trigger the header desync, so it's worth pinning down the exact shape before fixing.

revdiff doesn't tokenize markdown itself, it hands the file to chroma's markdown lexer. I reproduced the token stream for your exact repro:

- **Shape**: `SomeType { fieldA: X|Y, fieldB:
  Z, fieldC?: Int }`; more prose here.

## Next Section

on the current build ## Next Section still comes out as a heading token and keeps its color. The code span's closing backtick sits on line 2, before the header, so nothing downstream desyncs.

what is wrong in that repro: inside a list item the line-spanning code span isn't recognized as a code span at all. Chroma's list rule scopes inline matching to a single line, so the opening backtick is orphaned and the whole thing renders as plain text with no backtick styling. Cosmetic, no header desync.

the header desync is real but needs a different shape, the code span's closing backtick has to land after a header:

here is `an open span

## Swallowed Header

and it closes` here.

here chroma's code-span rule (`[^`]+`, where [^`] matches newlines) greedily eats everything up to the closing backtick, header included, so the header loses its color. Per commonmark a code span shouldn't cross a blank line, so chroma is over-matching across paragraph boundaries. A genuinely unclosed backtick doesn't desync, chroma leaves it as text and the next header stays colored.

so root cause is the vendored chroma markdown lexer, not revdiff's own parsing. The clean fix belongs upstream at https://github.com/alecthomas/chroma. If you can put together a minimal snippet that actually reproduces the desync (the "closing backtick after a header" shape above does it) and file it there, that's the right home for it. I may add a local workaround in revdiff in the meantime, a custom lexer that stops code spans at a blank line.

[denysshnurenko]

Thanks for tracing this down — filed the corrected repro upstream: alecthomas/chroma#1324. Also found that #1267 over there (similar symptom) was closed as "completed" but the same regex at markdown.go:102 is still on HEAD, so flagged that mismatch too.

[denysshnurenko]

I found that also (```) multiline code snippet does not work. Three backticks are identified as 2 separate code-spans. The second of them is not closed inline and is closed on following lines, so next header is broken

[umputun]

root cause is confirmed in chroma's markdown lexer, not revdiff: the inline code-span rule `[^`]+` (markdown.go:101 in vendored v2.27.0, :102 on chroma HEAD) lets [^]` match newlines, so a span runs past a blank line and swallows the next header's token. The triple-backtick case in the last comment is the same rule: chroma only treats ``` as a fence at line start with the closing fence on its own line, so anything else drops through to that greedy inline rule.

the catch: there's no upstream fix yet. Chroma converts these to discussions rather than tracking them as issues (chroma#1324 is now discussion alecthomas/chroma#1325, chroma#1267 is #1303), and the greedy regex is still untouched on HEAD and in the v3 alpha, so there's nothing to pick up from a version bump today.

revdiff hands markdown to chroma's lexer instead of tokenizing it itself, and reimplementing markdown tokenizing to work around a single lexer rule isn't worth carrying. Closing as out of scope. If chroma ever fixes the rule, a version bump picks it up for free.