codegen/string_escapes_test doesn't work with CFE

[johnniwinther]

The dart2js unittest codegen/string_escapes_test doesn't work with CFE.

[anders-sandholm]

Not sure if dart2js or front-end issue. Why not both?

[johnniwinther]

Good call - we need to coordinate the solution.

[rakudrama]

One of the errors is a source string containing \u2028\u2029. This shows up in the generated code as (), which is U+0028 U+0029.
So it seems that the code units are being truncated to the low byte somewhere, perhaps by storing code units into a Uint8List.

If the input is changed to \u2028\u2029\u20c1 then the output becomes ()\ufffd. This indicates that the low 8 bits are being interpreted as UTF-8, with 0xc1 being mapped to the error replacement character U+FFFD.

[lrhn]

If you look in source_file_helper, requesting a binary file for something supplied as a string to the memory source files does:

case InputKind.binary:
        if (source is String) {
          utf8SourceFiles[resourceUri] =
              new StringSourceFile.fromUri(resourceUri, source);
          source = source.codeUnits;
        }
        input =
            binarySourceFiles[resourceUri] = new Binary(resourceUri, source);
        break;

If the string is not Latin-1, this puts 16-bit integers into the Binary input. I'm guessing it's being truncated when read from that.

When you fix that, the test fails "correctly", meaning that the test is wrong.

The current specification says that the grapheme clusters CR+LF, CR or LF are line terminators, and that a line terminator introduces a single LF (U+000A) character into the multiline string.

The test says that:

await compileExpression("''' \n\r\u2028\u2029'''").then((String generated) {
      Expect.isTrue(
          generated.contains(r'"\r\u2028\u2029"') ||
              generated.contains(r"'\r\u2028\u2029'"),
          generated);
    });

That is, it expects a CR (\r) in the output, but the CR in the source is converted to a LF in the string, so the expectation should be r'"\n\u2028\u2029"' (or the same single quoted).

So, fix: https://dart-review.googlesource.com/c/sdk/+/108408