diff options
| author | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-27 18:45:44 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-27 18:45:44 -0700 |
| commit | 009b0e37e7c284c531cc3c44d0e5b5b1476f1666 (patch) | |
| tree | 3983dc05e6daea8276e2b954c3b5992f142dc07c /docs/configuration.md | |
| parent | 7bcb08cef5e6438ce8c8efa3da3a8f94f2a1fbf9 (diff) | |
| download | spark-009b0e37e7c284c531cc3c44d0e5b5b1476f1666.tar.gz spark-009b0e37e7c284c531cc3c44d0e5b5b1476f1666.tar.bz2 spark-009b0e37e7c284c531cc3c44d0e5b5b1476f1666.zip | |
Added an option to compress blocks in the block store
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 17 |
1 files changed, 17 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 4e47ca16e8..e4d5b21a12 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -112,6 +112,23 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td>spark.blockManager.compress</td> + <td>false</td> + <td> + Set to "true" to have Spark compress map output files, RDDs that get cached on disk, + and RDDs that get cached in serialized form. Generally a good idea when dealing with + large datasets, but might add some CPU overhead. + </td> +</tr> +<tr> + <td>spark.broadcast.compress</td> + <td>false</td> + <td> + Set to "true" to have Spark compress broadcast variables before sending them. + Generally a good idea when broadcasting large values. + </td> +</tr> +<tr> <td>spark.storage.memoryFraction</td> <td>0.66</td> <td> |