diff options
| author | Tathagata Das <tathagata.das1565@gmail.com> | 2013-01-01 12:21:32 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2013-01-01 12:21:32 -0800 |
| commit | 02497f0cd49a24ebc8b92d3471de250319fe56cd (patch) | |
| tree | 380c9b0f813ae9f01e087919ce2b7ce8a2005187 /docs/configuration.md | |
| parent | 18b9b3b99fd753899d19bd10f0dbef7d5c4ae8d7 (diff) | |
| download | spark-02497f0cd49a24ebc8b92d3471de250319fe56cd.tar.gz spark-02497f0cd49a24ebc8b92d3471de250319fe56cd.tar.bz2 spark-02497f0cd49a24ebc8b92d3471de250319fe56cd.zip | |
Updated Streaming Programming Guide.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index d8317ea97c..87cb4a6797 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -219,6 +219,17 @@ Apart from these, the following properties are also available, and may be useful Port for the master to listen on. </td> </tr> +<tr> + <td>spark.cleaner.delay</td> + <td>(disable)</td> + <td> + Duration (minutes) of how long Spark will remember any metadata (stages generated, tasks generated, etc.). + Periodic cleanups will ensure that metadata older than this duration will be forgetten. This is + useful for running Spark for many hours / days (for example, running 24/7 in case of Spark Streaming + applications). Note that any RDD that persists in memory for more than this duration will be cleared as well. + </td> +</tr> + </table> # Configuring Logging |