diff options
| author | Reynold Xin <rxin@apache.org> | 2014-07-15 01:46:57 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-07-15 01:46:57 -0700 |
| commit | dd95abada78b4d0aec97dacda50fdfd74464b073 (patch) | |
| tree | 823d5955b61420b64144dbf7fabee893492c1f13 /docs/configuration.md | |
| parent | 7446f5ff93142d2dd5c79c63fa947f47a1d4db8b (diff) | |
| download | spark-dd95abada78b4d0aec97dacda50fdfd74464b073.tar.gz spark-dd95abada78b4d0aec97dacda50fdfd74464b073.tar.bz2 spark-dd95abada78b4d0aec97dacda50fdfd74464b073.zip | |
[SPARK-2399] Add support for LZ4 compression.
Based on Greg Bowyer's patch from JIRA https://issues.apache.org/jira/browse/SPARK-2399
Author: Reynold Xin <rxin@apache.org>
Closes #1416 from rxin/lz4 and squashes the following commits:
6c8fefe [Reynold Xin] Fixed typo.
8a14d38 [Reynold Xin] [SPARK-2399] Add support for LZ4 compression.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 07aa4c0354..19fd980e60 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -350,7 +350,15 @@ Apart from these, the following properties are also available, and may be useful <td>32768</td> <td> Block size (in bytes) used in Snappy compression, in the case when Snappy compression codec - is used. + is used. Lowering this block size will also lower shuffle memory usage when Snappy is used. + </td> +</tr> +<tr> + <td><code>spark.io.compression.lz4.block.size</code></td> + <td>32768</td> + <td> + Block size (in bytes) used in LZ4 compression, in the case when LZ4 compression codec + is used. Lowering this block size will also lower shuffle memory usage when LZ4 is used. </td> </tr> <tr> |