diff options
| author | Jagadeesan <as2@us.ibm.com> | 2016-09-14 09:03:16 +0100 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-09-14 09:03:16 +0100 |
| commit | def7c265f539f3e119f068b6e9050300d05b14a4 (patch) | |
| tree | 1f868415685fa09748a1fb010d9918e9f6fbc78b | |
| parent | a454a4d86bbed1b6988da0a0e23b3e87a1a16340 (diff) | |
| download | spark-def7c265f539f3e119f068b6e9050300d05b14a4.tar.gz spark-def7c265f539f3e119f068b6e9050300d05b14a4.tar.bz2 spark-def7c265f539f3e119f068b6e9050300d05b14a4.zip | |
[SPARK-17449][DOCUMENTATION] Relation between heartbeatInterval and…
## What changes were proposed in this pull request?
The relation between spark.network.timeout and spark.executor.heartbeatInterval should be mentioned in the document.
… network timeout]
Author: Jagadeesan <as2@us.ibm.com>
Closes #15042 from jagadeesanas2/SPARK-17449.
| -rw-r--r-- | core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala | 1 | ||||
| -rw-r--r-- | docs/configuration.md | 3 |
2 files changed, 3 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala b/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala index c3764ac671..5242ab6f55 100644 --- a/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala +++ b/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala @@ -32,6 +32,7 @@ import org.apache.spark.util._ * A heartbeat from executors to the driver. This is a shared message used by several internal * components to convey liveness or execution information for in-progress tasks. It will also * expire the hosts that have not heartbeated for more than spark.network.timeout. + * spark.executor.heartbeatInterval should be significantly less than spark.network.timeout. */ private[spark] case class Heartbeat( executorId: String, diff --git a/docs/configuration.md b/docs/configuration.md index ebd0aa796d..8aea74505e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -987,7 +987,8 @@ Apart from these, the following properties are also available, and may be useful <td>10s</td> <td>Interval between each executor's heartbeats to the driver. Heartbeats let the driver know that the executor is still alive and update it with metrics for in-progress - tasks.</td> + tasks. spark.executor.heartbeatInterval should be significantly less than + spark.network.timeout</td> </tr> <tr> <td><code>spark.files.fetchTimeout</code></td> |