diff options
| author | tgravescs <tgraves_cs@yahoo.com> | 2013-11-19 12:39:26 -0600 |
|---|---|---|
| committer | tgravescs <tgraves_cs@yahoo.com> | 2013-11-19 12:44:00 -0600 |
| commit | 4093e9393aef95793f2d1d77fd0bbe80c8bb8d11 (patch) | |
| tree | fc899a11dc2fd20ecadc947fd2e8fea44425d008 /docs | |
| parent | e2ebc3a9d8bca83bf842b134f2f056c1af0ad2be (diff) | |
| download | spark-4093e9393aef95793f2d1d77fd0bbe80c8bb8d11.tar.gz spark-4093e9393aef95793f2d1d77fd0bbe80c8bb8d11.tar.bz2 spark-4093e9393aef95793f2d1d77fd0bbe80c8bb8d11.zip | |
Impove Spark on Yarn Error handling
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/running-on-yarn.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 6fd1d0d150..4056e9c15d 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -37,6 +37,8 @@ System Properties: * 'spark.yarn.applicationMaster.waitTries', property to set the number of times the ApplicationMaster waits for the the spark master and then also the number of tries it waits for the Spark Context to be intialized. Default is 10. * 'spark.yarn.submit.file.replication', the HDFS replication level for the files uploaded into HDFS for the application. These include things like the spark jar, the app jar, and any distributed cache files/archives. * 'spark.yarn.preserve.staging.files', set to true to preserve the staged files(spark jar, app jar, distributed cache files) at the end of the job rather then delete them. +* 'spark.yarn.scheduler.heartbeat.interval-ms', the interval in ms in which the Spark application master heartbeats into the YARN ResourceManager. Default is 5 seconds. +* 'spark.yarn.max.worker.failures', the maximum number of worker failures before failing the application. Default is the number of workers requested times 2 with minimum of 3. # Launching Spark on YARN |