diff options
author | WangTaoTheTonic <barneystinson@aliyun.com> | 2015-01-16 09:16:56 -0800 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-01-16 09:16:56 -0800 |
commit | 2be82b1e66cd188456bbf1e5abb13af04d1629d5 (patch) | |
tree | ba62a9aeb1084739fd75ffd6d991992b70be3af9 /docs/configuration.md | |
parent | a79a9f923c47f2ce7da93cf0ecfe2b66fcb9fdd4 (diff) | |
download | spark-2be82b1e66cd188456bbf1e5abb13af04d1629d5.tar.gz spark-2be82b1e66cd188456bbf1e5abb13af04d1629d5.tar.bz2 spark-2be82b1e66cd188456bbf1e5abb13af04d1629d5.zip |
[SPARK-1507][YARN]specify # cores for ApplicationMaster
Based on top of changes in https://github.com/apache/spark/pull/3806.
https://issues.apache.org/jira/browse/SPARK-1507
`--driver-cores` and `spark.driver.cores` for all cluster modes and `spark.yarn.am.cores` for yarn client mode.
Author: WangTaoTheTonic <barneystinson@aliyun.com>
Author: WangTao <barneystinson@aliyun.com>
Closes #4018 from WangTaoTheTonic/SPARK-1507 and squashes the following commits:
01419d3 [WangTaoTheTonic] amend the args name
b255795 [WangTaoTheTonic] indet thing
d86557c [WangTaoTheTonic] some comments amend
43c9392 [WangTao] fix compile error
b39a100 [WangTao] specify # cores for ApplicationMaster
Diffstat (limited to 'docs/configuration.md')
-rw-r--r-- | docs/configuration.md | 15 |
1 files changed, 11 insertions, 4 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 673cdb371a..efbab40853 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -102,11 +102,10 @@ of the most common options to set are: </td> </tr> <tr> - <td><code>spark.executor.memory</code></td> - <td>512m</td> + <td><code>spark.driver.cores</code></td> + <td>1</td> <td> - Amount of memory to use per executor process, in the same format as JVM memory strings - (e.g. <code>512m</code>, <code>2g</code>). + Number of cores to use for the driver process, only in cluster mode. </td> </tr> <tr> @@ -118,6 +117,14 @@ of the most common options to set are: </td> </tr> <tr> + <td><code>spark.executor.memory</code></td> + <td>512m</td> + <td> + Amount of memory to use per executor process, in the same format as JVM memory strings + (e.g. <code>512m</code>, <code>2g</code>). + </td> +</tr> +<tr> <td><code>spark.driver.maxResultSize</code></td> <td>1g</td> <td> |