Skip to content

Commit e421072

Browse files
WangTaoTheTonictgravescs
authored andcommitted
[SPARK-3722][Docs]minor improvement and fix in docs
https://issues.apache.org/jira/browse/SPARK-3722 Author: WangTao <[email protected]> Closes apache#2579 from WangTaoTheTonic/docsWork and squashes the following commits: 6f91cec [WangTao] use more wording express 29d22fa [WangTao] delete the specified version link 34cb4ea [WangTao] Update running-on-yarn.md 4ee1a26 [WangTao] minor improvement and fix in docs
1 parent 825709a commit e421072

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

docs/configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Then, you can supply configuration values at runtime:
5252
--conf "spark.executor.extraJavaOptions=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps" myApp.jar
5353
{% endhighlight %}
5454

55-
The Spark shell and [`spark-submit`](cluster-overview.html#launching-applications-with-spark-submit)
55+
The Spark shell and [`spark-submit`](submitting-applications.html)
5656
tool support two ways to load configurations dynamically. The first are command line options,
5757
such as `--master`, as shown above. `spark-submit` can accept any Spark property using the `--conf`
5858
flag, but uses special flags for properties that play a part in launching the Spark application.

docs/running-on-yarn.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
3939
<td><code>spark.yarn.preserve.staging.files</code></td>
4040
<td>false</td>
4141
<td>
42-
Set to true to preserve the staged files (Spark jar, app jar, distributed cache files) at the end of the job rather then delete them.
42+
Set to true to preserve the staged files (Spark jar, app jar, distributed cache files) at the end of the job rather than delete them.
4343
</td>
4444
</tr>
4545
<tr>
@@ -159,7 +159,7 @@ For example:
159159
lib/spark-examples*.jar \
160160
10
161161

162-
The above starts a YARN client program which starts the default Application Master. Then SparkPi will be run as a child thread of Application Master. The client will periodically poll the Application Master for status updates and display them in the console. The client will exit once your application has finished running. Refer to the "Viewing Logs" section below for how to see driver and executor logs.
162+
The above starts a YARN client program which starts the default Application Master. Then SparkPi will be run as a child thread of Application Master. The client will periodically poll the Application Master for status updates and display them in the console. The client will exit once your application has finished running. Refer to the "Debugging your Application" section below for how to see driver and executor logs.
163163

164164
To launch a Spark application in yarn-client mode, do the same, but replace "yarn-cluster" with "yarn-client". To run spark-shell:
165165

@@ -181,7 +181,7 @@ In YARN terminology, executors and application masters run inside "containers".
181181

182182
yarn logs -applicationId <app ID>
183183

184-
will print out the contents of all log files from all containers from the given application.
184+
will print out the contents of all log files from all containers from the given application. You can also view the container log files directly in HDFS using the HDFS shell or API. The directory where they are located can be found by looking at your YARN configs (`yarn.nodemanager.remote-app-log-dir` and `yarn.nodemanager.remote-app-log-dir-suffix`).
185185

186186
When log aggregation isn't turned on, logs are retained locally on each machine under `YARN_APP_LOGS_DIR`, which is usually configured to `/tmp/logs` or `$HADOOP_HOME/logs/userlogs` depending on the Hadoop version and installation. Viewing logs for a container requires going to the host that contains them and looking in this directory. Subdirectories organize log files by application ID and container ID.
187187

0 commit comments

Comments
 (0)