From 35b2415fb32cea506c60d6f1f824ac194f4604f8 Mon Sep 17 00:00:00 2001 From: Mosharaf Chowdhury Date: Thu, 17 Oct 2013 13:14:12 -0700 Subject: Code styling. Updated doc. --- docs/configuration.md | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'docs/configuration.md') diff --git a/docs/configuration.md b/docs/configuration.md index 7940d41a27..c5900d0e09 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -319,6 +319,14 @@ Apart from these, the following properties are also available, and may be useful Should be greater than or equal to 1. Number of allowed retries = this value - 1. + + spark.broadcast.blockSize + 4096 + + Size of each piece of a block in kilobytes for TorrentBroadcastFactory. + Too large a value decreases parallelism during broadcast (makes it slower); however, if it is too small, BlockManager might take a performance hit. + + -- cgit v1.2.3 From 6b6283628553616efb414c15f6e3791815c12016 Mon Sep 17 00:00:00 2001 From: Patrick Wendell Date: Fri, 18 Oct 2013 23:08:44 -0700 Subject: Clarify compression property. Clarifies that this governs compression of internal data, not input data or output data. --- docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration.md') diff --git a/docs/configuration.md b/docs/configuration.md index 7940d41a27..3cf43ab305 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -149,7 +149,7 @@ Apart from these, the following properties are also available, and may be useful spark.io.compression.codec org.apache.spark.io.
LZFCompressionCodec - The compression codec class to use for various compressions. By default, Spark provides two + The codec used to compress internal data such as RDD partitions and shuffle outputs. By default, Spark provides two codecs: org.apache.spark.io.LZFCompressionCodec and org.apache.spark.io.SnappyCompressionCodec. -- cgit v1.2.3