Real-world code snippets which `libcst` fails to parse

[Zac-HD]

@DRMacIver and I have been on a bit of a bughunt lately: the following cases are accepted by Python 3.10's compile() function, but cause libcst 1.0.0 to raise an exception:

import libcst

def compiles(source: str):
    try:
        compile(source, "<string>", "exec")
        return True
    except Exception:
        return False

def libcst_parses(source):
    try:
        libcst.parse_module(source)
        return True
    except Exception:
        return False

for s in [
    # Assignment expression while indexing
    '_[_:=0]',

    # Complex f-string expression
    'f"{_:{_:}{_}}"',

    # Expecting indentation?
    '(\n \\\n)',
    '(\n    """\n""")',
    'if _:\n    """\n)"""',
    'if _:\n    ("""\n""")',
    'if _:\n     """\n  """',
    'if _:\n        """\n    """ ',

    # (legally) missing whitespace around keywords
    'lambda*_:_',
    'lambda**_:_',
    '_ if 0else _',
    'with _ as():_',
    '(a:=()for _ in _)',

    '_ if _ else""if _ else _',
    '_ if _ else()if _ else _',
    '(_ if _ else 0for _ in _)',
    '(_ if _ else""for _ in _)',
    '(_ if _ else()for _ in _)',

    '(lambda:()for _ in _)',
    'def _():return()if _ else _',
]:
    assert compiles(s), repr(s)
    assert not libcst_native_parses(s), repr(s)

Note that each was extracted from code in the wild, so there's already more user impact than from (say) #446.

I was hoping to finish this before 1.0 was released, but you're too fast 💖

[zsol]

OK I've got a fix for some of these, and have a pretty good idea about how to fix the remaining whitespace validation issues (similar to the lambda situation).

I haven't looked into the indentation issues yet.

About to head out on vacation, so if anyone wants to jump in on these, feel free to 😝