diff options
| author | Artur Sukhenko <artur.sukhenko@gmail.com> | 2016-11-16 15:08:01 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-11-16 15:08:01 -0800 |
| commit | 55589987be89ff78dadf44498352fbbd811a206e (patch) | |
| tree | f15651b6feac7a401997fb5b234ec7fde0e5fc20 /docs/running-on-yarn.md | |
| parent | 2ca8ae9aa1b29bf1f46d0b656d9885e438e67f53 (diff) | |
| download | spark-55589987be89ff78dadf44498352fbbd811a206e.tar.gz spark-55589987be89ff78dadf44498352fbbd811a206e.tar.bz2 spark-55589987be89ff78dadf44498352fbbd811a206e.zip | |
[YARN][DOC] Increasing NodeManager's heap size with External Shuffle Service
## What changes were proposed in this pull request?
Suggest users to increase `NodeManager's` heap size if `External Shuffle Service` is enabled as
`NM` can spend a lot of time doing GC resulting in shuffle operations being a bottleneck due to `Shuffle Read blocked time` bumped up.
Also because of GC `NodeManager` can use an enormous amount of CPU and cluster performance will suffer.
I have seen NodeManager using 5-13G RAM and up to 2700% CPU with `spark_shuffle` service on.
## How was this patch tested?
#### Added step 5:
Author: Artur Sukhenko <artur.sukhenko@gmail.com>
Closes #15906 from Devian-ua/nmHeapSize.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index cd18808681..fe0221ce7c 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -559,6 +559,8 @@ pre-packaged distribution. 1. In the `yarn-site.xml` on each node, add `spark_shuffle` to `yarn.nodemanager.aux-services`, then set `yarn.nodemanager.aux-services.spark_shuffle.class` to `org.apache.spark.network.yarn.YarnShuffleService`. +1. Increase `NodeManager's` heap size by setting `YARN_HEAPSIZE` (1000 by default) in `etc/hadoop/yarn-env.sh` +to avoid garbage collection issues during shuffle. 1. Restart all `NodeManager`s in your cluster. The following extra configuration options are available when the shuffle service is running on YARN: |