diff options
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 20 |
1 files changed, 14 insertions, 6 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index e4e4b8d516..08c6befaf3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1098,24 +1098,32 @@ Apart from these, the following properties are also available, and may be useful available on YARN mode. For more detail, see the description <a href="job-scheduling.html#dynamic-resource-allocation">here</a>. <br><br> - This requires the following configurations to be set: + This requires <code>spark.shuffle.service.enabled</code> to be set. + The following configurations are also relevant: <code>spark.dynamicAllocation.minExecutors</code>, <code>spark.dynamicAllocation.maxExecutors</code>, and - <code>spark.shuffle.service.enabled</code> + <code>spark.dynamicAllocation.initialExecutors</code> </td> </tr> <tr> <td><code>spark.dynamicAllocation.minExecutors</code></td> - <td>(none)</td> + <td>0</td> <td> - Lower bound for the number of executors if dynamic allocation is enabled (required). + Lower bound for the number of executors if dynamic allocation is enabled. </td> </tr> <tr> <td><code>spark.dynamicAllocation.maxExecutors</code></td> - <td>(none)</td> + <td>Integer.MAX_VALUE</td> + <td> + Upper bound for the number of executors if dynamic allocation is enabled. + </td> +</tr> +<tr> + <td><code>spark.dynamicAllocation.maxExecutors</code></td> + <td><code>spark.dynamicAllocation.minExecutors</code></td> <td> - Upper bound for the number of executors if dynamic allocation is enabled (required). + Initial number of executors to run if dynamic allocation is enabled. </td> </tr> <tr> |