diff options
| author | witgo <witgo@qq.com> | 2014-06-19 12:11:26 -0500 |
|---|---|---|
| committer | Thomas Graves <tgraves@apache.org> | 2014-06-19 12:11:26 -0500 |
| commit | bce0897bc6b0fc9bca5444dbe3a9e75523ad7481 (patch) | |
| tree | c0b87857704b2bb014c2bd3a3409d6749b9ee8bb /docs/running-on-yarn.md | |
| parent | 67fca189c944b8f8ba222bb471e343893031bd7b (diff) | |
| download | spark-bce0897bc6b0fc9bca5444dbe3a9e75523ad7481.tar.gz spark-bce0897bc6b0fc9bca5444dbe3a9e75523ad7481.tar.bz2 spark-bce0897bc6b0fc9bca5444dbe3a9e75523ad7481.zip | |
[SPARK-2051]In yarn.ClientBase spark.yarn.dist.* do not work
Author: witgo <witgo@qq.com>
Closes #969 from witgo/yarn_ClientBase and squashes the following commits:
8117765 [witgo] review commit
3bdbc52 [witgo] Merge branch 'master' of https://github.com/apache/spark into yarn_ClientBase
5261b6c [witgo] fix sys.props.get("SPARK_YARN_DIST_FILES")
e3c1107 [witgo] update docs
b6a9aa1 [witgo] merge master
c8b4554 [witgo] review commit
2f48789 [witgo] Merge branch 'master' of https://github.com/apache/spark into yarn_ClientBase
8d7b82f [witgo] Merge branch 'master' of https://github.com/apache/spark into yarn_ClientBase
1048549 [witgo] remove Utils.resolveURIs
871f1db [witgo] add spark.yarn.dist.* documentation
41bce59 [witgo] review commit
35d6fa0 [witgo] move to ClientArguments
55d72fc [witgo] Merge branch 'master' of https://github.com/apache/spark into yarn_ClientBase
9cdff16 [witgo] review commit
8bc2f4b [witgo] review commit
20e667c [witgo] Merge branch 'master' into yarn_ClientBase
0961151 [witgo] merge master
ce609fc [witgo] Merge branch 'master' into yarn_ClientBase
8362489 [witgo] yarn.ClientBase spark.yarn.dist.* do not work
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 20 |
1 files changed, 17 insertions, 3 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 4243ef480b..fecd8f2cc2 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -68,15 +68,29 @@ Most of the configs are the same for Spark on YARN as for other deployment modes </td> </tr> <tr> - <td><code>spark.yarn.executor.memoryOverhead</code></td> - <td>384</code></td> + <td><code>spark.yarn.dist.archives</code></td> + <td>(none)</td> + <td> + Comma separated list of archives to be extracted into the working directory of each executor. + </td> +</tr> +<tr> + <td><code>spark.yarn.dist.files</code></td> + <td>(none)</td> + <td> + Comma-separated list of files to be placed in the working directory of each executor. + <td> +</tr> +<tr> + <td><code>spark.yarn.executor.memoryOverhead</code></td> + <td>384</td> <td> The amount of off heap memory (in megabytes) to be allocated per executor. This is memory that accounts for things like VM overheads, interned strings, other native overheads, etc. </td> </tr> <tr> <td><code>spark.yarn.driver.memoryOverhead</code></td> - <td>384</code></td> + <td>384</td> <td> The amount of off heap memory (in megabytes) to be allocated per driver. This is memory that accounts for things like VM overheads, interned strings, other native overheads, etc. </td> |