fix: don't backslash-escape non-syntax characters (invalid escape with POSIX classes) - #317
Conversation
A POSIX class such as [[:space:]] compiles to \p{...}, which sets the u
flag on the generated regular expression. In that mode only syntax
characters may follow a backslash, so escaping `,` `-` `#` or whitespace
turned the pattern into a SyntaxError:
minimatch('foo', ',[[:space:]]')
SyntaxError: Invalid regular expression: /^\,[\p{Z}\t\r\n\v\f]$/u: Invalid escape
None of those characters are regular expression syntax outside a
character class, which is the only context regExpEscape output is used
in, so escaping them was never necessary. `-` is only special inside a
class, `,` only inside a {n,m} quantifier - unreachable here because the
braces are themselves escaped - and `#` and whitespace only under the x
flag, which JavaScript does not have.
The escape must also round-trip through unescape(), which reverses it by
dropping backslashes, so encoding these characters some other way (\u002c
and friends) would corrupt literal patterns. Dropping the escape keeps
that round-trip exact.
Also de-duplicates the copy of regExpEscape in index.ts.
Fixes isaacs#273
|
One more symptom of the same root cause, which I found after opening this and which affects a wider set of patterns than the crash does. When the offending character and the POSIX class are in different path portions, try {
this.regexp = new RegExp(re, [...flags].join(''))
/* c8 ignore start */
} catch {
// should be impossible
this.regexp = false
}So instead of throwing,
To be precise about the blast radius, since it would be easy to overstate: Only the |
Fixes #273.
Problem
Any pattern that puts a POSIX class in the same path portion as a
,,-,#or space throws instead of matching:The issue reports the comma, but it is not comma-specific. Sweeping every printable ASCII character through
`a${ch}b[[:digit:]]`on 10.2.6 gives four offenders:Space and
-are the ones likely to be hit in practice, since they turn up in ordinary filenames —minimatch('x', 'my file/[[:digit:]]')is fine, butminimatch('x', 'my file[[:digit:]]')throws.Root cause
[[:space:]]compiles to\p{Z}…, and\prequires theuflag, whichAST.toMMRegExpduly sets:Under
u, only syntax characters may follow a backslash — everything else is anIdentityEscapeerror. ButregExpEscapeescapes a much wider set (src/ast.ts:172):so
,becomes\,, and the pattern fails to compile. Theuflag only propagates within a path portion, which is whya,b/[[:digit:]]is fine whilea,b[[:digit:]]is not.Fix
Escape only what is actually regex syntax outside a character class, which is the sole context this output is used in (
re += regExpEscape(c)inast.ts, and the segment map inindex.ts— both append to the regex body):Dropping
,,-,#and\sis safe because none is regex syntax there:-is only special inside a class,,only inside a{n,m}quantifier — unreachable, since{and}are themselves still escaped — and#/whitespace only under thexflag, which JavaScript does not have. The repo already suspected as much; the assertion intest/defaults.jscarried anoxlint-disable … no-useless-escapefor exactly this.Why not encode them instead. My first attempt escaped them as
,etc., which is valid in both modes. It corrupts literal patterns: the parsed set is produced byunescape(), which reverses an escape by dropping the backslash, soa,bdegrades to the literalau002cbandnew Minimatch('a,b').setbecame[['au002cb']]. Only removing the escape keeps that round-trip exact — there's a regression test pinning it.This also de-duplicates
regExpEscape, which was defined identically in bothast.tsandindex.ts;index.tsnow imports it.Behaviour
Matching is unchanged — the escaped and unescaped forms are equivalent in non-
umode, so this only affects the generated source text:The four snapshot files change by exactly one thing — a removed backslash before
#or-. Every changed line, deduped:No structural change, and no anchor, class or quantifier is touched.
Tests
New
test/posix-class-escape.jscovers the reported pattern, every printable ASCII character against a POSIX class, literal matching for the four affected characters, that genuine syntax characters are still escaped (a.bmust not matchaxb,a{2}bmust not matchaab), and theunescaperound-trip.Verified fail-then-pass against a true baseline —
git show HEAD:src/ast.ts/index.tsrestored, rebuilt withtshy, and confirmed the rebuiltdistcontained none of the change before running:Full suite is green at 6352/6352 (that includes
test/redos.js, since this touches a regex). Snapshots regenerated withTAP_SNAPSHOT=1 tap.oxlint src testclean,prettier --checkclean.What I did not verify
I did not benchmark. The replacement character class is strictly smaller than the original, so I would expect no regression, but I have not measured it.