SPARK-1689: Spark application should die when removed by Master

scheduler.error() will mask the error if there are active tasks. Being removed is a cataclysmic event for Spark applications, and should probably be treated as such.

Author: Aaron Davidson <[email protected]>

Closes #832 from aarondav/i-love-u and squashes the following commits:

9f1200f [Aaron Davidson] SPARK-1689: Spark application should die when removed by Master

(cherry picked from commit b0ce22e)
Signed-off-by: Tathagata Das <[email protected]>

Author: aarondav

core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala

@@ -88,6 +88,8 @@ private[spark] class SparkDeploySchedulerBackend(
     if (!stopping) {
       logError("Application has been killed. Reason: " + reason)
       scheduler.error(reason)
+      // Ensure the application terminates, as we can no longer run jobs.
+      sc.stop()
     }
   }