Skip to content

Commit e5775c1

Browse files
committed
Update doc about how to view the logs on Web UI when yarn log aggregation is enabled
1 parent 111c055 commit e5775c1

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/running-on-yarn.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,9 @@ In YARN terminology, executors and application masters run inside "containers".
6868

6969
yarn logs -applicationId <app ID>
7070

71-
will print out the contents of all log files from all containers from the given application. You can also view the container log files directly in HDFS using the HDFS shell or API. The directory where they are located can be found by looking at your YARN configs (`yarn.nodemanager.remote-app-log-dir` and `yarn.nodemanager.remote-app-log-dir-suffix`).
71+
will print out the contents of all log files from all containers from the given application. You can also view the container log files directly in HDFS using the HDFS shell or API. The directory where they are located can be found by looking at your YARN configs (`yarn.nodemanager.remote-app-log-dir` and `yarn.nodemanager.remote-app-log-dir-suffix`). The logs are also available on Web UI. You need have both the Spark history server and the MR history server running and configure `yarn.log.server.url` in yarn-site.xml properly. The log url on the Spark history server UI will redirect you to the MR history server to show the aggregated logs.
7272

73-
When log aggregation isn't turned on, logs are retained locally on each machine under `YARN_APP_LOGS_DIR`, which is usually configured to `/tmp/logs` or `$HADOOP_HOME/logs/userlogs` depending on the Hadoop version and installation. Viewing logs for a container requires going to the host that contains them and looking in this directory. Subdirectories organize log files by application ID and container ID.
73+
When log aggregation isn't turned on, logs are retained locally on each machine under `YARN_APP_LOGS_DIR`, which is usually configured to `/tmp/logs` or `$HADOOP_HOME/logs/userlogs` depending on the Hadoop version and installation. Viewing logs for a container requires going to the host that contains them and looking in this directory. Subdirectories organize log files by application ID and container ID. The logs are also available on Web UI even without running the MR history server.
7474

7575
To review per-container launch environment, increase `yarn.nodemanager.delete.debug-delay-sec` to a
7676
large value (e.g. 36000), and then access the application cache through `yarn.nodemanager.local-dirs`

0 commit comments

Comments
 (0)