[SPARK-7249] Updated Hadoop dependencies due to inconsistency in the versions #5786

FavioVazquez · 2015-04-29T19:37:32Z

Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons.

Changes proposed by @vanzin resulting from previous pull-request #5783 that did not fixed the problem correctly.

Please let me know if this is the correct way of doing this, the comments of @vanzin are in the pull-request mentioned.

…fix blocking error when connecting to HDFS via the Hadoop Cloudera HDFS CDH5 (fix for 2.5.0-cdh5.3.3 version)

…ow the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons

AmplabJenkins · 2015-04-29T19:42:11Z

Can one of the admins verify this patch?

vanzin · 2015-04-29T19:43:48Z

pom.xml

-        <hbase.version>0.98.7-hadoop2</hbase.version>
-        <avro.mapred.classifier>hadoop2</avro.mapred.classifier>
-        <codehaus.jackson.version>1.9.13</codehaus.jackson.version>
+        <hadoop.version>1.2.0</hadoop.version>


I don't think 1.2.0 is a valid version. The previous value used to be 1.0.4.

None of this should actually affect the releases people use since they build with profiles explicitly enabled as desired. It's a good change of defaults, for hygiene though.

The story here is that Spark 1.2 was sorta accidentally released as depending (in Maven) on 2.2, and so that was expressed later, but this is the logical conclusion of that process. The default build which you'd get if approaching this as a developer is and should be a consistent Hadoop 2.2 build.

However more needs to change besides this. The release script has to set a "hadoop-1" profile in some cases now. The docs about building vs various Hadoop versions has to be updated too.

vanzin · 2015-04-29T19:45:09Z

This LGTM, just a small issue left behind. There's some cleanup that could be done in the other hadoop-2.x profiles now, since they contain mostly redundant stuff.

@srowen could you trigger tests for this? Thanks!

tgravescs · 2015-04-29T22:00:32Z

jenkins, test this please

AmplabJenkins · 2015-04-29T22:02:10Z

Merged build triggered.

AmplabJenkins · 2015-04-29T22:02:17Z

Merged build started.

SparkQA · 2015-04-29T22:03:00Z

Test build #31337 has started for PR 5786 at commit 7e9955d.

srowen · 2015-04-29T22:25:12Z

(PS update the title please per https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark )

SparkQA · 2015-04-29T23:41:57Z

Test build #31337 has finished for PR 5786 at commit 7e9955d.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.
This patch adds the following new dependencies:
- jackson-core-asl-1.9.13.jar
- jackson-jaxrs-1.9.13.jar
- jackson-mapper-asl-1.9.13.jar
- jackson-xc-1.9.13.jar
- protobuf-java-2.5.0.jar
This patch removes the following dependencies:
- jackson-core-asl-1.8.8.jar
- jackson-jaxrs-1.8.8.jar
- jackson-mapper-asl-1.8.8.jar
- jackson-xc-1.8.8.jar
- protobuf-java-2.4.1.jar

AmplabJenkins · 2015-04-29T23:42:02Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-04-29T23:42:02Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31337/
Test FAILed.

FavioVazquez · 2015-04-30T00:08:47Z

Can someone please explain me what failed? I'm kinda new to this, and I'm not sure what failed, I want to know to fix it and maybe create another pull request. Thanks

vanzin · 2015-04-30T00:10:07Z

You can click on the logs to see the failure; it's a pyspark unit tests, I'd be surprised if your change caused it.

FavioVazquez · 2015-04-30T00:15:51Z

In the link https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31337/, there is no failure in the tests, but in the console output the error thrown was this:

FAIL: test_count_by_value_and_window (main.WindowFunctionTests)

Traceback (most recent call last):
File "pyspark/streaming/tests.py", line 418, in test_count_by_value_and_window
self._test_func(input, func, expected)
File "pyspark/streaming/tests.py", line 133, in _test_func
self.assertEqual(expected, result)
AssertionError: Lists differ: [[1], [2], [3], [4], [5], [6], [6], [6], [6], [6]] != [[1], [2], [3], [4], [5], [6], [6], [6], [6]]

First list contains 1 additional elements.
First extra element 9:
[6]

[[1], [2], [3], [4], [5], [6], [6], [6], [6], [6]]
? -----
[[1], [2], [3], [4], [5], [6], [6], [6], [6]]

Ran 40 tests in 134.429s

FAILED (failures=1)
('timeout after', 20)
('timeout after', 20)
('timeout after', 20)
('timeout after', 5)
Had test failures; see logs.
[error] Got a return code of 255 on line 240 of the run-tests script.
Archiving unit tests logs...

So I'm not sure what happened.

srowen · 2015-04-30T07:52:55Z

Yes I imagine this is an unrelated failure, so let's try again. however I'm also sure that this change will require changes in a number of other places, like the docs, the release script, and the Jenkins config. We'll have to coordinate that a bit. Looping in a busy @pwendell -- better to get this in early for 1.4.0? feels like we should.

srowen · 2015-04-30T07:53:02Z

Jenkins, retest this please

AmplabJenkins · 2015-04-30T07:57:10Z

Merged build triggered.

AmplabJenkins · 2015-04-30T07:57:17Z

Merged build started.

SparkQA · 2015-04-30T07:58:48Z

Test build #31400 has started for PR 5786 at commit 7e9955d.

SparkQA · 2015-04-30T09:57:12Z

Test build #31400 has finished for PR 5786 at commit 7e9955d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch adds the following new dependencies:
- jackson-core-asl-1.9.13.jar
- jackson-jaxrs-1.9.13.jar
- jackson-mapper-asl-1.9.13.jar
- jackson-xc-1.9.13.jar
- jaxb-api-2.2.7.jar
- jaxb-core-2.2.7.jar
- jaxb-impl-2.2.7.jar
- pmml-agent-1.1.15.jar
- pmml-model-1.1.15.jar
- pmml-schema-1.1.15.jar
- protobuf-java-2.5.0.jar
This patch removes the following dependencies:
- activation-1.1.jar
- jackson-core-asl-1.8.8.jar
- jackson-jaxrs-1.8.8.jar
- jackson-mapper-asl-1.8.8.jar
- jackson-xc-1.8.8.jar
- jaxb-api-2.2.2.jar
- jaxb-impl-2.2.3-1.jar
- protobuf-java-2.4.1.jar

AmplabJenkins · 2015-04-30T09:57:17Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-04-30T09:57:17Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31400/
Test PASSed.

FavioVazquez · 2015-04-30T19:19:16Z

Hi, now that the test have passed what will happen now? I've read the documentation in https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-PullRequest but I'm not sure, will this be merge to master? Should I do something else? keep the PR open?

Thanks.

srowen · 2015-04-30T20:37:13Z

You can see my comments here? there is more to do on this change I think before it can be merged.

FavioVazquez · 2015-04-30T20:39:25Z

Oh I see. Thanks. Should I keep the PR open then?

srowen · 2015-04-30T20:40:17Z

Yes, you just push more changes to the branch.

AmplabJenkins · 2015-05-13T09:57:20Z

Merged build started.

SparkQA · 2015-05-13T09:57:38Z

Test build #32605 has started for PR 5786 at commit 379f50d.

SparkQA · 2015-05-13T11:47:05Z

Test build #32605 has finished for PR 5786 at commit 379f50d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-13T11:47:10Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-13T11:47:10Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32605/
Test PASSed.

srowen · 2015-05-14T09:04:04Z

dev/create-release/create-release.sh

-  build/mvn -DskipTests -Dhadoop.version=2.2.0 -Dyarn.version=2.2.0 \
-    -Dscala-2.11 -Pyarn -Phive -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl \
+  build/mvn -DskipTests -Pyarn -Phive \
+    -Dscala-2.11 -Pspark-ganglia-lgpl -Pkinesis-asl \


This still needs -Phadoop-2.2 but maybe I can add that on merge

FavioVazquez · 2015-05-14T09:14:49Z

Are you sure Sean? I could make the change and push it, but if is easier to make the change in the merge you tell me.

srowen · 2015-05-14T09:16:47Z

Go ahead if you have a moment; only if it's not much work.
Thanks for your perseverance. This ends up being a great change IMHO. It will go in shortly.

FavioVazquez · 2015-05-14T09:18:57Z

I'm happy to help, give me a sec and I'll push the changes

AmplabJenkins · 2015-05-14T09:27:11Z

Merged build triggered.

AmplabJenkins · 2015-05-14T09:27:20Z

Merged build started.

SparkQA · 2015-05-14T09:29:15Z

Test build #32700 has started for PR 5786 at commit 11670e5.

SparkQA · 2015-05-14T10:48:06Z

Test build #32700 has finished for PR 5786 at commit 11670e5.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-14T10:48:12Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-14T10:48:13Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32700/
Test FAILed.

FavioVazquez · 2015-05-14T10:51:42Z

This has happened before @srowen, I think this is again an unrelated fail. Could you ask jenkins to retest this please?

srowen · 2015-05-14T10:52:54Z

I'm sure it is as you only made a doc change but while we are waiting: Jenkins retest this please

AmplabJenkins · 2015-05-14T10:57:10Z

Merged build triggered.

AmplabJenkins · 2015-05-14T10:57:20Z

Merged build started.

SparkQA · 2015-05-14T10:59:09Z

Test build #32708 has started for PR 5786 at commit 11670e5.

SparkQA · 2015-05-14T12:49:51Z

Test build #32708 has finished for PR 5786 at commit 11670e5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-14T12:49:56Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-14T12:49:57Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32708/
Test PASSed.

FavioVazquez · 2015-05-14T14:20:18Z

All test passed @srowen. It was, as expected, an unrelated error. Is everything set now to merge this PR?

…versions Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons. Changes proposed by vanzin resulting from previous pull-request #5783 that did not fixed the problem correctly. Please let me know if this is the correct way of doing this, the comments of vanzin are in the pull-request mentioned. Author: FavioVazquez <[email protected]> Closes #5786 from FavioVazquez/update-hadoop-dependencies and squashes the following commits: 11670e5 [FavioVazquez] - Added missing instance of -Phadoop-2.2 in create-release.sh 379f50d [FavioVazquez] - Added instances of -Phadoop-2.2 in create-release.sh, run-tests, scalastyle and building-spark.md - Reconstructed docs to not ask users to rely on default behavior 3f9249d [FavioVazquez] Merge branch 'master' of https://github.com/apache/spark into update-hadoop-dependencies 31bdafa [FavioVazquez] - Added missing instances in -Phadoop-1 in create-release.sh, run-tests and in the building-spark documentation cbb93e8 [FavioVazquez] - Added comment related to SPARK-3710 about hadoop-yarn-server-tests in Hadoop 2.2 that fails to pull some needed dependencies 83dc332 [FavioVazquez] - Cleaned up the main POM concerning the yarn profile - Erased hadoop-2.2 profile from yarn/pom.xml and its content was integrated into yarn/pom.xml 93f7624 [FavioVazquez] - Deleted unnecessary comments and <activation> tag on the YARN profile in the main POM 668d126 [FavioVazquez] - Moved <dependencies> <activation> and <properties> sections of the hadoop-2.2 profile in the YARN POM to the YARN profile in the root POM - Erased unnecessary hadoop-2.2 profile from the YARN POM fda6a51 [FavioVazquez] - Updated hadoop1 releases in create-release.sh due to changes in the default hadoop version set - Erased unnecessary instance of -Dyarn.version=2.2.0 in create-release.sh - Prettify comment in yarn/pom.xml 0470587 [FavioVazquez] - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in create-release.sh - Updated how the releases are made in the create-release.sh no that the default hadoop version is the 2.2.0 - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in scalastyle - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in run-tests - Better example given in the hadoop-third-party-distributions.md now that the default hadoop version is 2.2.0 a650779 [FavioVazquez] - Default value of avro.mapred.classifier has been set to hadoop2 in pom.xml - Cleaned up hadoop-2.3 and 2.4 profiles due to change in the default set in avro.mapred.classifier in pom.xml 199f40b [FavioVazquez] - Erased unnecessary CDH5-specific note in docs/building-spark.md - Remove example of instance -Phadoop-2.2 -Dhadoop.version=2.2.0 in docs/building-spark.md - Enabled hadoop-2.2 profile when the Hadoop version is 2.2.0, which is now the default .Added comment in the yarn/pom.xml to specify that. 88a8b88 [FavioVazquez] - Simplified Hadoop profiles due to new setting of global properties in the pom.xml file - Added comment to specify that the hadoop-2.2 profile is now the default hadoop profile in the pom.xml file - Erased hadoop-2.2 from related hadoop profiles now that is a no-op in the make-distribution.sh file 70b8344 [FavioVazquez] - Fixed typo in the make-distribution.sh file and added hadoop-1 in the Related profiles 287fa2f [FavioVazquez] - Updated documentation about specifying the hadoop version in building-spark. Now is clear that Spark will build against Hadoop 2.2.0 by default. - Added Cloudera CDH 5.3.3 without MapReduce example in the building-spark doc. 1354292 [FavioVazquez] - Fixed hadoop-1 version to match jenkins build profile in hadoop1.0 tests and documentation 6b4bfaf [FavioVazquez] - Cleanup in hadoop-2.x profiles since they contained mostly redundant stuff. 7e9955d [FavioVazquez] - Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons 660decc [FavioVazquez] - Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons ec91ce3 [FavioVazquez] - Updated protobuf-java version of com.google.protobuf dependancy to fix blocking error when connecting to HDFS via the Hadoop Cloudera HDFS CDH5 (fix for 2.5.0-cdh5.3.3 version) (cherry picked from commit 7fb715d) Signed-off-by: Sean Owen <[email protected]>

…versions Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons. Changes proposed by vanzin resulting from previous pull-request apache#5783 that did not fixed the problem correctly. Please let me know if this is the correct way of doing this, the comments of vanzin are in the pull-request mentioned. Author: FavioVazquez <[email protected]> Closes apache#5786 from FavioVazquez/update-hadoop-dependencies and squashes the following commits: 11670e5 [FavioVazquez] - Added missing instance of -Phadoop-2.2 in create-release.sh 379f50d [FavioVazquez] - Added instances of -Phadoop-2.2 in create-release.sh, run-tests, scalastyle and building-spark.md - Reconstructed docs to not ask users to rely on default behavior 3f9249d [FavioVazquez] Merge branch 'master' of https://github.com/apache/spark into update-hadoop-dependencies 31bdafa [FavioVazquez] - Added missing instances in -Phadoop-1 in create-release.sh, run-tests and in the building-spark documentation cbb93e8 [FavioVazquez] - Added comment related to SPARK-3710 about hadoop-yarn-server-tests in Hadoop 2.2 that fails to pull some needed dependencies 83dc332 [FavioVazquez] - Cleaned up the main POM concerning the yarn profile - Erased hadoop-2.2 profile from yarn/pom.xml and its content was integrated into yarn/pom.xml 93f7624 [FavioVazquez] - Deleted unnecessary comments and <activation> tag on the YARN profile in the main POM 668d126 [FavioVazquez] - Moved <dependencies> <activation> and <properties> sections of the hadoop-2.2 profile in the YARN POM to the YARN profile in the root POM - Erased unnecessary hadoop-2.2 profile from the YARN POM fda6a51 [FavioVazquez] - Updated hadoop1 releases in create-release.sh due to changes in the default hadoop version set - Erased unnecessary instance of -Dyarn.version=2.2.0 in create-release.sh - Prettify comment in yarn/pom.xml 0470587 [FavioVazquez] - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in create-release.sh - Updated how the releases are made in the create-release.sh no that the default hadoop version is the 2.2.0 - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in scalastyle - Erased unnecessary instance of -Phadoop-2.2 -Dhadoop.version=2.2.0 in run-tests - Better example given in the hadoop-third-party-distributions.md now that the default hadoop version is 2.2.0 a650779 [FavioVazquez] - Default value of avro.mapred.classifier has been set to hadoop2 in pom.xml - Cleaned up hadoop-2.3 and 2.4 profiles due to change in the default set in avro.mapred.classifier in pom.xml 199f40b [FavioVazquez] - Erased unnecessary CDH5-specific note in docs/building-spark.md - Remove example of instance -Phadoop-2.2 -Dhadoop.version=2.2.0 in docs/building-spark.md - Enabled hadoop-2.2 profile when the Hadoop version is 2.2.0, which is now the default .Added comment in the yarn/pom.xml to specify that. 88a8b88 [FavioVazquez] - Simplified Hadoop profiles due to new setting of global properties in the pom.xml file - Added comment to specify that the hadoop-2.2 profile is now the default hadoop profile in the pom.xml file - Erased hadoop-2.2 from related hadoop profiles now that is a no-op in the make-distribution.sh file 70b8344 [FavioVazquez] - Fixed typo in the make-distribution.sh file and added hadoop-1 in the Related profiles 287fa2f [FavioVazquez] - Updated documentation about specifying the hadoop version in building-spark. Now is clear that Spark will build against Hadoop 2.2.0 by default. - Added Cloudera CDH 5.3.3 without MapReduce example in the building-spark doc. 1354292 [FavioVazquez] - Fixed hadoop-1 version to match jenkins build profile in hadoop1.0 tests and documentation 6b4bfaf [FavioVazquez] - Cleanup in hadoop-2.x profiles since they contained mostly redundant stuff. 7e9955d [FavioVazquez] - Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons 660decc [FavioVazquez] - Updated Hadoop dependencies due to inconsistency in the versions. Now the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons ec91ce3 [FavioVazquez] - Updated protobuf-java version of com.google.protobuf dependancy to fix blocking error when connecting to HDFS via the Hadoop Cloudera HDFS CDH5 (fix for 2.5.0-cdh5.3.3 version)

FavioVazquez added 3 commits April 29, 2015 13:28

- Updated protobuf-java version of com.google.protobuf dependancy to …

ec91ce3

…fix blocking error when connecting to HDFS via the Hadoop Cloudera HDFS CDH5 (fix for 2.5.0-cdh5.3.3 version)

- Updated Hadoop dependencies due to inconsistency in the versions. N…

660decc

…ow the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons

- Updated Hadoop dependencies due to inconsistency in the versions. N…

7e9955d

…ow the global properties are the ones used by the hadoop-2.2 profile, and the profile was set to empty but kept for backwards compatibility reasons

vanzin reviewed Apr 29, 2015
View reviewed changes

FavioVazquez changed the title ~~Updated Hadoop dependencies due to inconsistency in the versions~~ [SPARK-7249] Updated Hadoop dependencies due to inconsistency in the versions Apr 29, 2015

srowen reviewed May 14, 2015
View reviewed changes

- Added missing instance of -Phadoop-2.2 in create-release.sh

11670e5

asfgit closed this in 7fb715d May 14, 2015

[SPARK-7249] Updated Hadoop dependencies due to inconsistency in the versions #5786

[SPARK-7249] Updated Hadoop dependencies due to inconsistency in the versions #5786

Uh oh!

Conversation

FavioVazquez commented Apr 29, 2015

Uh oh!

AmplabJenkins commented Apr 29, 2015

Uh oh!

vanzin Apr 29, 2015

Choose a reason for hiding this comment

Uh oh!

srowen Apr 29, 2015

Choose a reason for hiding this comment

Uh oh!

vanzin commented Apr 29, 2015

Uh oh!

tgravescs commented Apr 29, 2015

Uh oh!

AmplabJenkins commented Apr 29, 2015

Uh oh!

AmplabJenkins commented Apr 29, 2015

Uh oh!

SparkQA commented Apr 29, 2015

Uh oh!

srowen commented Apr 29, 2015

Uh oh!

SparkQA commented Apr 29, 2015

Uh oh!

AmplabJenkins commented Apr 29, 2015

Uh oh!

AmplabJenkins commented Apr 29, 2015

Uh oh!

FavioVazquez commented Apr 30, 2015

Uh oh!

vanzin commented Apr 30, 2015

Uh oh!

FavioVazquez commented Apr 30, 2015

FAIL: test_count_by_value_and_window (main.WindowFunctionTests)

Uh oh!

srowen commented Apr 30, 2015

Uh oh!

srowen commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

SparkQA commented Apr 30, 2015

Uh oh!

SparkQA commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

AmplabJenkins commented Apr 30, 2015

Uh oh!

FavioVazquez commented Apr 30, 2015

Uh oh!

srowen commented Apr 30, 2015

Uh oh!

FavioVazquez commented Apr 30, 2015

Uh oh!

srowen commented Apr 30, 2015

Uh oh!

AmplabJenkins commented May 13, 2015

Uh oh!

SparkQA commented May 13, 2015

Uh oh!

SparkQA commented May 13, 2015

Uh oh!

AmplabJenkins commented May 13, 2015

Uh oh!

AmplabJenkins commented May 13, 2015

Uh oh!

srowen May 14, 2015

Choose a reason for hiding this comment

Uh oh!

FavioVazquez commented May 14, 2015

Uh oh!

srowen commented May 14, 2015

Uh oh!