diff options
| author | CrazyJvm <crazyjvm@gmail.com> | 2014-07-08 13:55:42 -0500 |
|---|---|---|
| committer | Thomas Graves <tgraves@apache.org> | 2014-07-08 13:55:42 -0500 |
| commit | b520b6453ed76926108e0bdd56114d16e1d86850 (patch) | |
| tree | a106e1705c70ee00050ff88f8cf28f51b5a38f1a /docs/running-on-yarn.md | |
| parent | c8a2313cdf825e0191680a423d17619b5504ff89 (diff) | |
| download | spark-b520b6453ed76926108e0bdd56114d16e1d86850.tar.gz spark-b520b6453ed76926108e0bdd56114d16e1d86850.tar.bz2 spark-b520b6453ed76926108e0bdd56114d16e1d86850.zip | |
SPARK-2400 : fix spark.yarn.max.executor.failures explaination
According to
```scala
private val maxNumExecutorFailures = sparkConf.getInt("spark.yarn.max.executor.failures",
sparkConf.getInt("spark.yarn.max.worker.failures", math.max(args.numExecutors * 2, 3)))
```
default value should be numExecutors * 2, with minimum of 3, and it's same to the config
`spark.yarn.max.worker.failures`
Author: CrazyJvm <crazyjvm@gmail.com>
Closes #1282 from CrazyJvm/yarn-doc and squashes the following commits:
1a5f25b [CrazyJvm] remove deprecated config
c438aec [CrazyJvm] fix style
86effa6 [CrazyJvm] change expression
211f130 [CrazyJvm] fix html tag
2900d23 [CrazyJvm] fix style
a4b2e27 [CrazyJvm] fix configuration spark.yarn.max.executor.failures
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 5d8d603aa3..0362f5a223 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -55,7 +55,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes </tr> <tr> <td><code>spark.yarn.max.executor.failures</code></td> - <td>2*numExecutors</td> + <td>numExecutors * 2, with minimum of 3</td> <td> The maximum number of executor failures before failing the application. </td> |