diff options
| author | Devaraj K <devaraj@apache.org> | 2016-04-05 14:12:00 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-04-05 14:12:00 -0500 |
| commit | bc36df127d3b9f56b4edaeb5eca7697d4aef761a (patch) | |
| tree | 4382521ace7488fe00e7f8b1eb49a6c4f056a9e3 /docs/running-on-yarn.md | |
| parent | 463bac001171622538fc93d2e31d1a617ab562e6 (diff) | |
| download | spark-bc36df127d3b9f56b4edaeb5eca7697d4aef761a.tar.gz spark-bc36df127d3b9f56b4edaeb5eca7697d4aef761a.tar.bz2 spark-bc36df127d3b9f56b4edaeb5eca7697d4aef761a.zip | |
[SPARK-13063][YARN] Make the SPARK YARN STAGING DIR as configurable
## What changes were proposed in this pull request?
Made the SPARK YARN STAGING DIR as configurable with the configuration as 'spark.yarn.staging-dir'.
## How was this patch tested?
I have verified it manually by running applications on yarn, If the 'spark.yarn.staging-dir' is configured then the value used as staging directory otherwise uses the default value i.e. file system’s home directory for the user.
Author: Devaraj K <devaraj@apache.org>
Closes #12082 from devaraj-kavali/SPARK-13063.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index bb83272ec8..ddc75a70b9 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -160,6 +160,13 @@ If you need a reference to the proper location to put log files in the YARN so t </td> </tr> <tr> + <td><code>spark.yarn.stagingDir</code></td> + <td>Current user's home directory in the filesystem</td> + <td> + Staging directory used while submitting applications. + </td> +</tr> +<tr> <td><code>spark.yarn.preserve.staging.files</code></td> <td><code>false</code></td> <td> |