diff options
| author | Andrew Or <andrew@databricks.com> | 2015-02-06 10:54:23 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-02-06 10:55:13 -0800 |
| commit | fe3740c4c859d087b714c666741a29061bba5f58 (patch) | |
| tree | d88dd2b9c1407861441aa0554161793f994b78a0 /docs | |
| parent | 1a88f20de798030a7d5713bd267f612ba5617fca (diff) | |
| download | spark-fe3740c4c859d087b714c666741a29061bba5f58.tar.gz spark-fe3740c4c859d087b714c666741a29061bba5f58.tar.bz2 spark-fe3740c4c859d087b714c666741a29061bba5f58.zip | |
[SPARK-5636] Ramp up faster in dynamic allocation
A recent patch #4051 made the initial number default to 0. With this change, any Spark application using dynamic allocation's default settings will ramp up very slowly. Since we never request more executors than needed to saturate the pending tasks, it is safe to ramp up quickly. The current default of 60 may be too slow.
Author: Andrew Or <andrew@databricks.com>
Closes #4409 from andrewor14/dynamic-allocation-interval and squashes the following commits:
d3cc485 [Andrew Or] Lower request interval
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 4c86cb7c16..00e973c245 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1140,7 +1140,7 @@ Apart from these, the following properties are also available, and may be useful </tr> <tr> <td><code>spark.dynamicAllocation.schedulerBacklogTimeout</code></td> - <td>60</td> + <td>5</td> <td> If dynamic allocation is enabled and there have been pending tasks backlogged for more than this duration (in seconds), new executors will be requested. For more detail, see this |