Skip to content

Commit 6b3403b

Browse files
author
Andrew Or
committed
Scope all RDD methods
This commit provides a mechanism to set and unset the call scope around each RDD operation defined in RDD.scala. This is useful for tagging an RDD with the scope in which it is created. This will be extended to similar methods in SparkContext.scala and other relevant files in a future commit.
1 parent 55f553a commit 6b3403b

File tree

2 files changed

+208
-107
lines changed

2 files changed

+208
-107
lines changed

core/src/main/scala/org/apache/spark/SparkContext.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1983,10 +1983,10 @@ object SparkContext extends Logging {
19831983
}
19841984

19851985
private[spark] val SPARK_JOB_DESCRIPTION = "spark.job.description"
1986-
19871986
private[spark] val SPARK_JOB_GROUP_ID = "spark.jobGroup.id"
1988-
19891987
private[spark] val SPARK_JOB_INTERRUPT_ON_CANCEL = "spark.job.interruptOnCancel"
1988+
private[spark] val RDD_SCOPE_KEY = "spark.rdd.scope"
1989+
private[spark] val RDD_SCOPE_NO_OVERRIDE_KEY = "spark.rdd.scope.noOverride"
19901990

19911991
/**
19921992
* Executor id for the driver. In earlier versions of Spark, this was `<driver>`, but this was

0 commit comments

Comments
 (0)