Is there an existing issue for this?
Describe the bug
We got a bug report in gitea that a file with HTTP response was partially displayed.
Chroma does handle it correctly (as per screenshot) but the tokeniser skips html tokens if lexer isn't wrapped in coalesce one.
With coalesce:
without (no html part):
Unsure if it's a bug or a our mistake and lexer should've always been wrapped there, but from the description of it, it seems like there shouldn't be such difference between outputs depending on it's use. Figured I'll file this to double check just in case it's not expected.
To Reproduce
Not able to do either here since it's a library usage. Snippet that reproduces the issue is below, it uses the copy of quick.Highlight with added if to gate coalesce:
package main
import (
"fmt"
"io"
"os"
"github.com/alecthomas/chroma/v2"
"github.com/alecthomas/chroma/v2/formatters"
"github.com/alecthomas/chroma/v2/lexers"
"github.com/alecthomas/chroma/v2/styles"
)
func main() {
code:=`HTTP/1.0 400 Bad request
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>`
Highlight(os.Stdout, code, "http", "html", "monokai",false)
fmt.Println("-------------------")
Highlight(os.Stdout, code, "http", "html", "monokai",true)
}
// Highlight some text.
//
// Lexer, formatter and style may be empty, in which case a best-effort is made.
func Highlight(w io.Writer, source, lexer, formatter, style string,coalesce bool) error {
// Determine lexer.
l := lexers.Get(lexer)
if l == nil {
l = lexers.Analyse(source)
}
if l == nil {
l = lexers.Fallback
}
if coalesce {
l = chroma.Coalesce(l)
}
// Determine formatter.
f := formatters.Get(formatter)
if f == nil {
f = formatters.Fallback
}
// Determine style.
s := styles.Get(style)
if s == nil {
s = styles.Fallback
}
it, err := l.Tokenise(nil, source)
if err != nil {
return err
}
return f.Format(w, s, it)
}
Is there an existing issue for this?
Describe the bug
We got a bug report in gitea that a file with HTTP response was partially displayed.
Chroma does handle it correctly (as per screenshot) but the tokeniser skips html tokens if lexer isn't wrapped in coalesce one.
With coalesce:
without (no html part):
Unsure if it's a bug or a our mistake and lexer should've always been wrapped there, but from the description of it, it seems like there shouldn't be such difference between outputs depending on it's use. Figured I'll file this to double check just in case it's not expected.
To Reproduce
Not able to do either here since it's a library usage. Snippet that reproduces the issue is below, it uses the copy of quick.Highlight with added if to gate coalesce: