diff options
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index cb0c65e2d2..dac8bb1d52 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -197,6 +197,15 @@ Apart from these, the following properties are also available, and may be useful Spark's dependencies and user dependencies. It is currently an experimental feature. </td> </tr> +<tr> + <td><code>spark.python.worker.memory</code></td> + <td>512m</td> + <td> + Amount of memory to use per python worker process during aggregation, in the same + format as JVM memory strings (e.g. <code>512m</code>, <code>2g</code>). If the memory + used during aggregation goes above this amount, it will spill the data into disks. + </td> +</tr> </table> #### Shuffle Behavior |