diff options
| author | Reynold Xin <rxin@apache.org> | 2014-08-12 22:50:29 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-08-12 22:50:29 -0700 |
| commit | 676f98289dad61c091bb45bd35a2b9613b22d64a (patch) | |
| tree | 56195950714160e2f9d52b6557f1560068a89ea4 /docs | |
| parent | c235b83e2782cce0626ecc403c0a67e442be52c1 (diff) | |
| download | spark-676f98289dad61c091bb45bd35a2b9613b22d64a.tar.gz spark-676f98289dad61c091bb45bd35a2b9613b22d64a.tar.bz2 spark-676f98289dad61c091bb45bd35a2b9613b22d64a.zip | |
[SPARK-2953] Allow using short names for io compression codecs
Instead of requiring "org.apache.spark.io.LZ4CompressionCodec", it is easier for users if Spark just accepts "lz4", "lzf", "snappy".
Author: Reynold Xin <rxin@apache.org>
Closes #1873 from rxin/compressionCodecShortForm and squashes the following commits:
9f50962 [Reynold Xin] Specify short-form compression codec names first.
63f78ee [Reynold Xin] Updated configuration documentation.
47b3848 [Reynold Xin] [SPARK-2953] Allow using short names for io compression codecs
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 617a72a021..8136bd62ab 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -373,10 +373,12 @@ Apart from these, the following properties are also available, and may be useful </tr> <tr> <td><code>spark.io.compression.codec</code></td> - <td>org.apache.spark.io.<br />SnappyCompressionCodec</td> + <td>snappy</td> <td> - The codec used to compress internal data such as RDD partitions and shuffle outputs. - By default, Spark provides three codecs: <code>org.apache.spark.io.LZ4CompressionCodec</code>, + The codec used to compress internal data such as RDD partitions and shuffle outputs. By default, + Spark provides three codecs: <code>lz4</code>, <code>lzf</code>, and <code>snappy</code>. You + can also use fully qualified class names to specify the codec, e.g. + <code>org.apache.spark.io.LZ4CompressionCodec</code>, <code>org.apache.spark.io.LZFCompressionCodec</code>, and <code>org.apache.spark.io.SnappyCompressionCodec</code>. </td> |