diff options
| author | Ryan Williams <ryan.blake.williams@gmail.com> | 2015-10-19 16:34:15 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-10-19 16:34:15 -0700 |
| commit | 16906ef23a7aa2854c8cdcaa3bb3808ab39e0eec (patch) | |
| tree | 5ae77f9fd48113e8052d6a34dfbcff42fdaf611a /core | |
| parent | fc26f32cf1bede8b9a1343dca0c0182107c9985e (diff) | |
| download | spark-16906ef23a7aa2854c8cdcaa3bb3808ab39e0eec.tar.gz spark-16906ef23a7aa2854c8cdcaa3bb3808ab39e0eec.tar.bz2 spark-16906ef23a7aa2854c8cdcaa3bb3808ab39e0eec.zip | |
[SPARK-11120] Allow sane default number of executor failures when dynamically allocating in YARN
I also added some information to container-failure error msgs about what host they failed on, which would have helped me identify the problem that lead me to this JIRA and PR sooner.
Author: Ryan Williams <ryan.blake.williams@gmail.com>
Closes #9147 from ryan-williams/dyn-exec-failures.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/SparkConf.scala | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkConf.scala b/core/src/main/scala/org/apache/spark/SparkConf.scala index 1a0ac3d017..58d3b846fd 100644 --- a/core/src/main/scala/org/apache/spark/SparkConf.scala +++ b/core/src/main/scala/org/apache/spark/SparkConf.scala @@ -595,7 +595,9 @@ private[spark] object SparkConf extends Logging { "spark.rpc.lookupTimeout" -> Seq( AlternateConfig("spark.akka.lookupTimeout", "1.4")), "spark.streaming.fileStream.minRememberDuration" -> Seq( - AlternateConfig("spark.streaming.minRememberDuration", "1.5")) + AlternateConfig("spark.streaming.minRememberDuration", "1.5")), + "spark.yarn.max.executor.failures" -> Seq( + AlternateConfig("spark.yarn.max.worker.failures", "1.5")) ) /** |