SPARK-1291: Link the spark UI to RM ui in yarn-client mode #1112

witgo · 2014-06-18T09:02:40Z

No description provided.

AmplabJenkins · 2014-06-18T09:04:50Z

Merged build triggered.

AmplabJenkins · 2014-06-18T09:04:59Z

Merged build started.

AmplabJenkins · 2014-06-18T09:45:28Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-06-18T09:45:29Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15869/

tgravescs · 2014-06-23T14:28:27Z

yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorLauncher.scala

-    logInfo("Registering the ApplicationMaster")
-    // TODO: Find out client's Spark UI address and fill in here?
-    amClient.registerApplicationMaster(Utils.localHostName(), 0, "")
+    val appUIAddress = sparkConf.getOption("spark.driver.appUIAddress").getOrElse("")


You can just use sparkConf.get("spark.driver.appUIAddress", "")

AmplabJenkins · 2014-06-23T16:30:15Z

Merged build triggered.

AmplabJenkins · 2014-06-23T16:30:24Z

Merged build started.

AmplabJenkins · 2014-06-23T17:13:21Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-06-23T17:13:22Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16030/

AmplabJenkins · 2014-06-24T03:15:13Z

Merged build triggered.

AmplabJenkins · 2014-06-24T03:15:18Z

Merged build started.

AmplabJenkins · 2014-06-24T04:08:37Z

Merged build finished.

AmplabJenkins · 2014-06-24T04:08:37Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16047/

AmplabJenkins · 2014-06-25T02:55:17Z

Merged build triggered.

AmplabJenkins · 2014-06-25T02:55:27Z

Merged build started.

AmplabJenkins · 2014-06-25T03:05:32Z

Merged build finished.

AmplabJenkins · 2014-06-25T03:05:32Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16099/

witgo · 2014-06-25T03:07:02Z

Jenkins, test this please.

AmplabJenkins · 2014-06-25T03:10:18Z

Merged build triggered.

AmplabJenkins · 2014-06-25T03:10:27Z

Merged build started.

AmplabJenkins · 2014-06-25T04:06:39Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-06-25T04:06:40Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16100/

SparkQA · 2014-07-15T15:02:56Z

QA tests have started for PR 1112. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16680/consoleFull

witgo · 2014-07-15T15:04:56Z

@tgravescs The code has been submitted. Because I don't have the hadoop 0.23.x cluster, the code no strict test.

SparkQA · 2014-07-15T15:17:47Z

QA tests have started for PR 1112. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16681/consoleFull

tgravescs · 2014-07-15T16:05:31Z

Thanks, I'll test out the 0.23 code.

SparkQA · 2014-07-15T16:39:15Z

QA results for PR 1112:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
case class AddWebUIFilter(filterName:String, filterParams: String, proxyBase :String)

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16680/consoleFull

SparkQA · 2014-07-15T16:51:13Z

QA results for PR 1112:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
case class AddWebUIFilter(filterName:String, filterParams: String, proxyBase :String)

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16681/consoleFull

tgravescs · 2014-07-15T17:01:33Z

Jenkins, test this please

SparkQA · 2014-07-15T17:08:00Z

QA tests have started for PR 1112. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16687/consoleFull

SparkQA · 2014-07-15T18:44:07Z

QA results for PR 1112:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds the following public classes (experimental):
case class AddWebUIFilter(filterName:String, filterParams: String, proxyBase :String)

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16687/consoleFull

tgravescs · 2014-07-15T18:53:49Z

Looks good. Thanks @witgo

Author: witgo <[email protected]> Closes apache#1112 from witgo/SPARK-1291 and squashes the following commits: 6022bcd [witgo] review commit 1fbb925 [witgo] add addAmIpFilter to yarn alpha 210299c [witgo] review commit 1b92a07 [witgo] review commit 6896586 [witgo] Add comments to addWebUIFilter 3e9630b [witgo] review commit 142ee29 [witgo] review commit 1fe7710 [witgo] Link the spark UI to RM ui in yarn-client mode Conflicts: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorLauncher.scala

The issue is caused by #1112 . Author: GuoQiang Li <[email protected]> Closes #1501 from witgo/webui_style and squashes the following commits: 4b34998 [GuoQiang Li] In some cases, pages display incorrect in WebUI

Author: witgo <[email protected]> Closes apache#1112 from witgo/SPARK-1291 and squashes the following commits: 6022bcd [witgo] review commit 1fbb925 [witgo] add addAmIpFilter to yarn alpha 210299c [witgo] review commit 1b92a07 [witgo] review commit 6896586 [witgo] Add comments to addWebUIFilter 3e9630b [witgo] review commit 142ee29 [witgo] review commit 1fe7710 [witgo] Link the spark UI to RM ui in yarn-client mode

The issue is caused by apache#1112 . Author: GuoQiang Li <[email protected]> Closes apache#1501 from witgo/webui_style and squashes the following commits: 4b34998 [GuoQiang Li] In some cases, pages display incorrect in WebUI

…verride option (apache#1112) [EZSPA-1073] Add ssibility to work with spark.rapids.shims-provider-override option --------- Co-authored-by: Egor Krivokon <>

rahulsinghaliitd mentioned this pull request Jun 23, 2014

SPARK-2150: Provide direct link to finished application UI in yarn resou... #1094

Closed

tgravescs reviewed Jun 23, 2014
View reviewed changes

witgo added 3 commits July 15, 2014 22:49

review commit

1b92a07

review commit

210299c

add addAmIpFilter to yarn alpha

1fbb925

review commit

6022bcd

asfgit closed this in 72ea56d Jul 15, 2014

witgo mentioned this pull request Jul 20, 2014

[YARN][SPARK-2606]:In some cases,the spark UI pages display incorrect #1501

Closed

SPARK-1291: Link the spark UI to RM ui in yarn-client mode #1112

SPARK-1291: Link the spark UI to RM ui in yarn-client mode #1112

Uh oh!

Conversation

witgo commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

tgravescs Jun 23, 2014

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented Jun 23, 2014

Uh oh!

AmplabJenkins commented Jun 23, 2014

Uh oh!

AmplabJenkins commented Jun 23, 2014

Uh oh!

AmplabJenkins commented Jun 23, 2014

Uh oh!

AmplabJenkins commented Jun 24, 2014

Uh oh!

AmplabJenkins commented Jun 24, 2014

Uh oh!

AmplabJenkins commented Jun 24, 2014

Uh oh!

AmplabJenkins commented Jun 24, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

witgo commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

AmplabJenkins commented Jun 25, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

witgo commented Jul 15, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

tgravescs commented Jul 15, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

tgravescs commented Jul 15, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

SparkQA commented Jul 15, 2014

Uh oh!

tgravescs commented Jul 15, 2014

Uh oh!

Uh oh!