VerifyError when overriding a class that overrides a vararg method and makes it final, depending on compilation order

[provegard]

(Copied from here.)

Consider the following three Scala source files:

JSFunction.scala:

import jdk.nashorn.api.scripting.AbstractJSObject

abstract class JSFunction extends AbstractJSObject {
  final override def call(thiz: scala.Any, args: AnyRef*): AnyRef = doCall(thiz, args: _*)

  def doCall(thiz: scala.Any, args: AnyRef*): AnyRef
}

InvokableWrapper.scala:

class InvokableWrapper extends JSFunction {
  override def doCall(thiz: Any, args: AnyRef*): AnyRef = {
    println("Called with: " + args.mkString(", "))
    null
  }
}

Main.scala:

object Main extends App {
  val wrapper = new InvokableWrapper
  wrapper.call(null, "a", "b")
}

This is what I do:

> scalac *.scala
> scala Main

I get:

java.lang.VerifyError: class InvokableWrapper overrides final method call.(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at scala.reflect.internal.util.ScalaClassLoader.run(ScalaClassLoader.scala:94)
        at scala.reflect.internal.util.ScalaClassLoader.run$(ScalaClassLoader.scala:90)
        at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:129)
        at scala.tools.nsc.CommonRunner.run(ObjectRunner.scala:22)
        at scala.tools.nsc.CommonRunner.run$(ObjectRunner.scala:21)
        at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
        at scala.tools.nsc.CommonRunner.runAndCatch(ObjectRunner.scala:29)
        at scala.tools.nsc.CommonRunner.runAndCatch$(ObjectRunner.scala:28)
        at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
        at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:61)
        at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:88)
        at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:99)
        at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:104)
        at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

If I remove final in JSFunction.scala, I get the expected output:

Called with: a, b

For reference:

> scalac -version
Scala code runner version 2.12.1 -- Copyright 2002-2016, LAMP/EPFL and Lightbend, Inc.

> java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

I'm on Windows 10.

More information:

• If I put the JSFunction and InvokableWrapper classes in the same file, I don't get the error.
• If I compile like @lrytz did, with scalac JSFunction.scala InvokableWrapper.scala Main.scala, then it works. If I compile with scalac *.scala, it doesn't work.

In the actual code base, this error happens whenever a specific source file is changed and sbt or IDEA performs incremental compilation, so this is a real problem that cannot be fixed by compiling in a certain order. The only workaround I know of is to do a full rebuild.

[provegard]

Compiling in alphabetical order fails: scalac InvokableWrapper.scala JSFunction.scala Main.scala

[lrytz]

Wow, indeed, I can reproduce it on Mac OS.

[lrytz]

Indeed InvokableWrapper gets an override in one case

diff a/InvokableWrapper.asm b/InvokableWrapper.asm
44,63d43
<   // access flags 0x1011
<   public final synthetic call(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
<     // parameter final  thiz
<     // parameter final  args
<    L0
<     LINENUMBER 1 L0
<     ALOAD 0
...

[lrytz]

It's a bug in varargs bridge generation. Will take a look tomorrow.

[provegard]

Great, thanks!

[lrytz]

Minimized:

$ cat A.java
public class A {
  public Object m(Object... os) { return null; }
}
$ cat Test.scala
class C extends B
class B extends A {
  final override def m(os: Object*): Object = null
}

$ javac A.java
$ scalac Test.scala && scala -e "new C"
java.lang.VerifyError: class C overrides final method m.([Ljava/lang/Object;)Ljava/lang/Object;

[lrytz]

Hmm, not so straightforward to fix. Consider

A.java

public interface A {
  public Object m(Object... os);
}

Test.scala

class C extends BB
class BB extends B with A
class B { final def m(os: Object*): Object = null }

In refchecks, we first see if C needs a varargs bridge.

• It inherits a (Scala) varargs method m from B
• It also inherits a matching Java varargs method from A, and it doesn't inherit a varargs bridge (because BB did not yet go through refchecks -- this depends on the order of declarations in the source file, or the order of source files if they are in separate files)

So a bridge is added to C. Then refchecks runs on BB and also adds a bridge to it. Flags are copied, so the bridge in BB is final, and the overriding version in C is illegal.

How could we decide while looking at C that it should not get a varargs bridge, because one of its parents will get it? Not sure..

An easy fix would be to remove the final flag from varargs bridges. But that would still leave us in a situation where the generated classfiles depend on the order of declarations / order of source files passed to the compiler.

[provegard]

I suppose user code cannot override a varags bridge since it's synthetic (or whatever the correct term is - artifact?), so removing final sounds like it shouldn't have any side effects, correct?

But the last part about order of declarations/source files - do you mean that if BB is compiled before C, then BB will get a bridge but C won't, whereas if C is compiled before BB then both will get bridges? If so, can you generate a bridge for C even if it's redundant, just for the sake of consistency?

[lrytz]

From discussion with @adriaanm

• Do the transformation in a phase's info transformer (instead of the tree transformer), then the order can be enforced by forcing a symbol's type
• Check how solve the issue for ordinary bridges
• The transformation should move to a different place, doing it in RefChecks is strange. Maybe erasure, that's where we do ordinary bridges.