diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-09-18 13:48:41 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-09-18 13:48:41 -0700 |
| commit | 348d7c9a93dd00d3d1859342a8eb0aea2e77f597 (patch) | |
| tree | 95d9a65d2e95aedd3eb5f31ac41e779e79aa8d6d /docs | |
| parent | 3a22b1004f527d54d399dd0225cd7f2f8ffad9c5 (diff) | |
| download | spark-348d7c9a93dd00d3d1859342a8eb0aea2e77f597.tar.gz spark-348d7c9a93dd00d3d1859342a8eb0aea2e77f597.tar.bz2 spark-348d7c9a93dd00d3d1859342a8eb0aea2e77f597.zip | |
[SPARK-9808] Remove hash shuffle file consolidation.
Author: Reynold Xin <rxin@databricks.com>
Closes #8812 from rxin/SPARK-9808-1.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 10 |
1 files changed, 0 insertions, 10 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 1a701f1888..3700051efb 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -391,16 +391,6 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> - <td><code>spark.shuffle.consolidateFiles</code></td> - <td>false</td> - <td> - If set to "true", consolidates intermediate files created during a shuffle. Creating fewer - files can improve filesystem performance for shuffles with large numbers of reduce tasks. It - is recommended to set this to "true" when using ext4 or xfs filesystems. On ext3, this option - might degrade performance on machines with many (>8) cores due to filesystem limitations. - </td> -</tr> -<tr> <td><code>spark.shuffle.file.buffer</code></td> <td>32k</td> <td> |