[SPARK-18136] Fix SPARK_JARS_DIR for Python pip install on Windows

jsnowacki · HyukjinKwon · commit 0b3e7cc6ac29 · 2017-09-23T21:05:04.000+09:00
## What changes were proposed in this pull request? Fix for setup of `SPARK_JARS_DIR` on Windows as it looks for `%SPARK_HOME%\RELEASE` file instead of `%SPARK_HOME%\jars` as it should. RELEASE file is not included in the `pip` build of PySpark. ## How was this patch tested? Local install of PySpark on Anaconda 4.4.0 (Python 3.6.1). Author: Jakub Nowacki <j.s.nowacki@gmail.com> Closes #19310 from jsnowacki/master. (cherry picked from commit c11f24a) Signed-off-by: hyukjinkwon <gurwls223@gmail.com>
diff --git a/bin/spark-class2.cmd b/bin/spark-class2.cmd
@@ -29,7 +29,7 @@ if "x%1"=="x" (
 )
 
 rem Find Spark jars.
-if exist "%SPARK_HOME%\RELEASE" (
+if exist "%SPARK_HOME%\jars" (
   set SPARK_JARS_DIR="%SPARK_HOME%\jars"
 ) else (
   set SPARK_JARS_DIR="%SPARK_HOME%\assembly\target\scala-%SPARK_SCALA_VERSION%\jars"

Original file line number	Diff line number	Diff line change
`@@ -29,7 +29,7 @@ if "x%1"=="x" (`
`29`	`29`	`)`
`30`	`30`
`31`	`31`	`rem Find Spark jars.`
`32`		`-if exist "%SPARK_HOME%\RELEASE" (`
	`32`	`+if exist "%SPARK_HOME%\jars" (`
`33`	`33`	`set SPARK_JARS_DIR="%SPARK_HOME%\jars"`
`34`	`34`	`) else (`
`35`	`35`	`set SPARK_JARS_DIR="%SPARK_HOME%\assembly\target\scala-%SPARK_SCALA_VERSION%\jars"`