diff options
| author | Andrew Or <andrew@databricks.com> | 2015-05-22 17:37:38 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-05-22 17:37:38 -0700 |
| commit | 3d8760d76eae41dcaab8e9aeda19619f3d5f1596 (patch) | |
| tree | 9d6296496875ac1b3a1fec38e050c5cdaab5c324 /docs/configuration.md | |
| parent | 3c1305107a2d6d2de862e8b41dbad0e85585b1ef (diff) | |
| download | spark-3d8760d76eae41dcaab8e9aeda19619f3d5f1596.tar.gz spark-3d8760d76eae41dcaab8e9aeda19619f3d5f1596.tar.bz2 spark-3d8760d76eae41dcaab8e9aeda19619f3d5f1596.zip | |
[SPARK-7771] [SPARK-7779] Dynamic allocation: lower default timeouts further
The default add time of 5s is still too slow for small jobs. Also, the current default remove time of 10 minutes seem rather high. This patch lowers both and rephrases a few log messages.
Author: Andrew Or <andrew@databricks.com>
Closes #6301 from andrewor14/da-minor and squashes the following commits:
6d614a6 [Andrew Or] Lower log level
2811492 [Andrew Or] Log information when requests are canceled
5fcd3eb [Andrew Or] Fix tests
3320710 [Andrew Or] Lower timeouts + rephrase a few log messages
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 0de824546c..30508a617f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1194,7 +1194,7 @@ Apart from these, the following properties are also available, and may be useful </tr> <tr> <td><code>spark.dynamicAllocation.executorIdleTimeout</code></td> - <td>600s</td> + <td>60s</td> <td> If dynamic allocation is enabled and an executor has been idle for more than this duration, the executor will be removed. For more detail, see this @@ -1224,7 +1224,7 @@ Apart from these, the following properties are also available, and may be useful </tr> <tr> <td><code>spark.dynamicAllocation.schedulerBacklogTimeout</code></td> - <td>5s</td> + <td>1s</td> <td> If dynamic allocation is enabled and there have been pending tasks backlogged for more than this duration, new executors will be requested. For more detail, see this |