Skip to content

Commit dffb5dd

Browse files
committed
[SPARK-2627] download pep8 at runtime
See the discussion here: apache#1744 (comment) Get the pep8 utility at runtime so that it’s not required to be installed on the build server.
1 parent a1ce7ae commit dffb5dd

File tree

1 file changed

+13
-1
lines changed

1 file changed

+13
-1
lines changed

dev/lint-python

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,21 @@ SPARK_ROOT_DIR="$(dirname $SCRIPT_DIR)"
2222
PEP8_REPORT_PATH="$SPARK_ROOT_DIR/dev/pep8-report.txt"
2323

2424
cd $SPARK_ROOT_DIR
25+
26+
# See: https://github.com/apache/spark/pull/1744#issuecomment-50982162
27+
# Get pep8 at runtime so that we don't rely on it being installed on the build server.
28+
# TODOs:
29+
# - Dynamically determine latest release version of pep8 and use that.
30+
# - Download this from a more reliable source. (GitHub raw can be flaky, apparently. (?))
31+
PEP8_SCRIPT_PATH="$SPARK_ROOT_DIR/dev/pep8.py"
32+
curl --silent -o "$PEP8_SCRIPT_PATH" \
33+
"https://raw.githubusercontent.com/jcrocholl/pep8/1.5.7/pep8.py"
34+
2535
# There is no need to write this output to a file
2636
#+ first, but we do so so that the check status can
2737
#+ be output before the report, like with the
2838
#+ scalastyle and RAT checks.
29-
pep8 ./python --exclude="cloudpickle.py" \
39+
python $PEP8_SCRIPT_PATH ./python --exclude="cloudpickle.py" \
3040
> "$PEP8_REPORT_PATH"
3141
pep8_status=${PIPESTATUS[0]} #$?
3242

@@ -39,4 +49,6 @@ if [ $pep8_status -ne 0 ]
3949
fi
4050

4151
rm -f "$PEP8_REPORT_PATH"
52+
rm "$PEP8_SCRIPT_PATH"
53+
4254
exit $pep8_status

0 commit comments

Comments
 (0)