diff options
| author | Andrew Or <andrewor14@gmail.com> | 2014-01-10 15:45:38 -0800 |
|---|---|---|
| committer | Andrew Or <andrewor14@gmail.com> | 2014-01-10 15:45:38 -0800 |
| commit | 2e393cd5fdfbf3a85fced370b5c42315e86dad49 (patch) | |
| tree | dc09bd89d83f58c0b7f518007094c76db2befcc0 | |
| parent | e4c51d21135978908f7f4a46683f70ef98b720ec (diff) | |
| download | spark-2e393cd5fdfbf3a85fced370b5c42315e86dad49.tar.gz spark-2e393cd5fdfbf3a85fced370b5c42315e86dad49.tar.bz2 spark-2e393cd5fdfbf3a85fced370b5c42315e86dad49.zip | |
Update documentation for externalSorting
| -rw-r--r-- | docs/configuration.md | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index c1158491f0..40a57c4bc6 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -391,9 +391,8 @@ Apart from these, the following properties are also available, and may be useful <td>spark.shuffle.externalSorting</td> <td>true</td> <td> - If set to "true", spills in-memory maps used for shuffles to disk when a memory threshold is reached. This - threshold is specified by <code>spark.shuffle.memoryFraction</code>. Enable this especially for memory-intensive - applications. + If set to "true", limits the amount of memory used during reduces by spilling data out to disk. This spilling + threshold is specified by <code>spark.shuffle.memoryFraction</code>. </td> </tr> <tr> |