Skip to content

Commit 235590c

Browse files
authored
feat: add JSONL support to JSON lexer (#1262)
### Description This PR adds support for JSON Lines (JSONL) to the existing JSON lexer. Because the existing JSON lexer already handles repeating objects natively, this implementation simply maps the \`jsonl\` file extension, alias, and mime type to the \`JSON\` lexer configuration without needing any underlying state machine changes. Inspired by walles/moor#407. ### Changes * **Alias added:** \`jsonl\` * **File extension added:** \`*.jsonl\` * **MIME type added:** \`application/jsonl\` * **Tests:** Added \`jsonl.actual\` and \`jsonl.expected\` validation files to ensure tokens are correctly extracted across newlines in \`.jsonl\` files. ### Testing Ran \`go test ./lexers -run TestLexers\` locally to verify that JSON Lines objects format accurately and consistently.
1 parent f9b5c97 commit 235590c

4 files changed

Lines changed: 23 additions & 0 deletions

File tree

lexers/embedded/json.xml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,15 @@
22
<config>
33
<name>JSON</name>
44
<alias>json</alias>
5+
<alias>jsonl</alias>
56
<filename>*.json</filename>
7+
<filename>*.jsonl</filename>
68
<filename>*.jsonc</filename>
79
<filename>*.json5</filename>
810
<filename>*.avsc</filename>
911
<filename>.luaurc</filename>
1012
<mime_type>application/json</mime_type>
13+
<mime_type>application/jsonl</mime_type>
1114
<dot_all>true</dot_all>
1215
<not_multiline>true</not_multiline>
1316
</config>

lexers/testdata/jsonl.actual

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
{"name": "Alice"}
2+
{"name": "Bob"}

lexers/testdata/jsonl.expected

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
[
2+
{"type":"Punctuation","value":"{"},
3+
{"type":"NameTag","value":"\"name\""},
4+
{"type":"Punctuation","value":":"},
5+
{"type":"Text","value":" "},
6+
{"type":"LiteralStringDouble","value":"\"Alice\""},
7+
{"type":"Punctuation","value":"}"},
8+
{"type":"Text","value":"\n"},
9+
{"type":"Punctuation","value":"{"},
10+
{"type":"NameTag","value":"\"name\""},
11+
{"type":"Punctuation","value":":"},
12+
{"type":"Text","value":" "},
13+
{"type":"LiteralStringDouble","value":"\"Bob\""},
14+
{"type":"Punctuation","value":"}"},
15+
{"type":"Text","value":"\n"}
16+
]

test.jsonl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
{"name": "Alice"}
2+
{"name": "Bob"}

0 commit comments

Comments
 (0)