diff options
| author | Yu Peng <loneknightpy@gmail.com> | 2016-10-18 13:23:31 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-10-18 13:23:31 -0700 |
| commit | 231f39e3f6641953a90bc4c40444ede63f363b23 (patch) | |
| tree | 679bf48c57a3e00196e9f610d1bc0bf386bcd97f /docs/spark-standalone.md | |
| parent | 37686539f546ac7a3657dbfc59b7ac982b4b9bce (diff) | |
| download | spark-231f39e3f6641953a90bc4c40444ede63f363b23.tar.gz spark-231f39e3f6641953a90bc4c40444ede63f363b23.tar.bz2 spark-231f39e3f6641953a90bc4c40444ede63f363b23.zip | |
[SPARK-17711] Compress rolled executor log
## What changes were proposed in this pull request?
This PR adds support for executor log compression.
## How was this patch tested?
Unit tests
cc: yhuai tdas mengxr
Author: Yu Peng <loneknightpy@gmail.com>
Closes #15285 from loneknightpy/compress-executor-log.
Diffstat (limited to 'docs/spark-standalone.md')
| -rw-r--r-- | docs/spark-standalone.md | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/docs/spark-standalone.md b/docs/spark-standalone.md index 7b82b957d5..1c0b60f7b9 100644 --- a/docs/spark-standalone.md +++ b/docs/spark-standalone.md @@ -250,6 +250,15 @@ SPARK_WORKER_OPTS supports the following system properties: especially if you run jobs very frequently. </td> </tr> +<tr> + <td><code>spark.worker.ui.compressedLogFileLengthCacheSize</code></td> + <td>100</td> + <td> + For compressed log files, the uncompressed file can only be computed by uncompressing the files. + Spark caches the uncompressed file size of compressed log files. This property controls the cache + size. + </td> +</tr> </table> # Connecting an Application to the Cluster |