Write down source compatibility guidelines between Scala dialects

[Blaisorblade]

Again and again, we want to merge pull requests which change the Scala source language.

We should evolve precise guidelines for how to handle them and where to draw the line.
I think changing the source language is the point of the exercise, but of course we need good changes.

Guidelines to decide what's in and what's out are hard, that's why I say "evolve"; but already now we should have some guidelines on how to do the breaking changes we want.

1. We should require good documentation for users, possibly of better quality than the existing Scalac documentation. I wouldn't rule out altogether forking the spec, though that's probably a hard sell unless somebody volunteers for that.
2. Of course, we should check for compatibility with the community build. Over time, we'll have to figure out what kind of breakage we accept.

[puffnfresh]

Here's the first guideline, from the blog post:

Other fixes, whilst affecting semantics, would do so only in a conservative way – programs which are valid when compiled with the Typesafe Scala compiler would have the same meaning and binary representation when compiled with the fixes in place.

[djspiewak]

Proposed generic informal goals:

1. Any project source that compiles and runs under scalac should compile and run under tlc with the same set of flags. Two differences are allowed: warnings issued, and classpath (e.g. tlc may require some classpath entries that scalac does not, or vice versa)
2. Sources that compile under both scalac and tlc with the same flags should have the same semantics.
3. Any project binaries produced by tlc should load cleanly from a scalac project and should be callable without FFI. In other words, tlc should produce Scala APIs. The exact bytecode metadata may be distinct, so long as it is compatible.

The first and second items should be easily testable just using projects in the Scala community (similar to what Typesafe does with its compatibility tests). The third item is somewhat harder and may require specific tests, but probably eased through the same sort of mass continuous integration.

[non]

It's possible that (1) is slightly too conservative. For example, consider the following valid Scala program:

\u006f\u0062\u006a\u0065\u0063\u0074\u0020\u004d\u0061\u0069\u006e\u0020\u007b
\u0020\u0020\u0064\u0065\u0066\u0020\u006d\u0061\u0069\u006e\u0028\u0061\u0072\u0067\u0073\u003a\u0020\u0041\u0072\u0072\u0061\u0079\u005b\u0053\u0074\u0072\u0069\u006e\u0067\u005d\u0029\u0020\u007b
\u0020\u0020\u0020\u0020\u0070\u0072\u0069\u006e\u0074\u006c\u006e\u0028\u0022\u0068\u0065\u006c\u006c\u006f\u0020\u0077\u006f\u0072\u006c\u0064\u0022\u0029
\u0020\u0020\u007d
\u007d

My proposed change to unicode escaping (#7) would make this currently valid Scala program fail to compile.

[puffnfresh]

@djspiewak @non well, now I'm conflicted.

[djspiewak]

@non For this reason, your Unicode stuff should be behind a flag. That is
implicit in my proposed restrictions.

[non]

Sure, possibly -Zsafe-unicode-escapes or similar.

[puffnfresh]

@non is it actually easy to preserve both behaviours?