diff options
author | peng.zhang <peng.zhang@xiaomi.com> | 2014-09-23 08:45:56 -0500 |
---|---|---|
committer | Thomas Graves <tgraves@apache.org> | 2014-09-23 08:45:56 -0500 |
commit | 14f8c340402366cb998c563b3f7d9ff7d9940271 (patch) | |
tree | 0349775622ba6a907aadad58df8d9f063ca9139f /yarn/alpha | |
parent | f9d6220c792b779be385f3022d146911a22c2130 (diff) | |
download | spark-14f8c340402366cb998c563b3f7d9ff7d9940271.tar.gz spark-14f8c340402366cb998c563b3f7d9ff7d9940271.tar.bz2 spark-14f8c340402366cb998c563b3f7d9ff7d9940271.zip |
[YARN] SPARK-2668: Add variable of yarn log directory for reference from the log4j configuration
Assign value of yarn container log directory to java opts "spark.yarn.app.container.log.dir", So user defined log4j.properties can reference this value and write log to YARN container's log directory.
Otherwise, user defined file appender will only write to container's CWD, and log files in CWD will not be displayed on YARN UIļ¼and either cannot be aggregated to HDFS log directory after job finished.
User defined log4j.properties reference example:
log4j.appender.rolling_file.File = ${spark.yarn.app.container.log.dir}/spark.log
Author: peng.zhang <peng.zhang@xiaomi.com>
Closes #1573 from renozhang/yarn-log-dir and squashes the following commits:
16c5cb8 [peng.zhang] Update doc
f2b5e2a [peng.zhang] Change variable's name, and update running-on-yarn.md
503ea2d [peng.zhang] Support log4j log to yarn container dir
Diffstat (limited to 'yarn/alpha')
0 files changed, 0 insertions, 0 deletions