diff options
| author | xukun 00228947 <xukun.xu@huawei.com> | 2015-02-26 13:24:00 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-02-26 13:24:00 -0800 |
| commit | 8942b522d8a3269a2a357e3a274ed4b3e66ebdde (patch) | |
| tree | e1f9a65a0784fb4a3c83ad344f7e853a554b1729 /docs/monitoring.md | |
| parent | 10094a523e3993b775111ae9b22ca31cc0d76e03 (diff) | |
| download | spark-8942b522d8a3269a2a357e3a274ed4b3e66ebdde.tar.gz spark-8942b522d8a3269a2a357e3a274ed4b3e66ebdde.tar.bz2 spark-8942b522d8a3269a2a357e3a274ed4b3e66ebdde.zip | |
[SPARK-3562]Periodic cleanup event logs
Author: xukun 00228947 <xukun.xu@huawei.com>
Closes #4214 from viper-kun/cleaneventlog and squashes the following commits:
7a5b9c5 [xukun 00228947] fix issue
31674ee [xukun 00228947] fix issue
6e3d06b [xukun 00228947] fix issue
373f3b9 [xukun 00228947] fix issue
71782b5 [xukun 00228947] fix issue
5b45035 [xukun 00228947] fix issue
70c28d6 [xukun 00228947] fix issues
adcfe86 [xukun 00228947] Periodic cleanup event logs
Diffstat (limited to 'docs/monitoring.md')
| -rw-r--r-- | docs/monitoring.md | 25 |
1 files changed, 24 insertions, 1 deletions
diff --git a/docs/monitoring.md b/docs/monitoring.md index 009a344dff..37ede476c1 100644 --- a/docs/monitoring.md +++ b/docs/monitoring.md @@ -86,7 +86,7 @@ follows: </td> </tr> <tr> - <td>spark.history.fs.updateInterval</td> + <td>spark.history.fs.update.interval.seconds</td> <td>10</td> <td> The period, in seconds, at which information displayed by this history server is updated. @@ -145,6 +145,29 @@ follows: If disabled, no access control checks are made. </td> </tr> + <tr> + <td>spark.history.fs.cleaner.enabled</td> + <td>false</td> + <td> + Specifies whether the History Server should periodically clean up event logs from storage. + </td> + </tr> + <tr> + <td>spark.history.fs.cleaner.interval.seconds</td> + <td>86400</td> + <td> + How often the job history cleaner checks for files to delete, in seconds. Defaults to 86400 (one day). + Files are only deleted if they are older than spark.history.fs.cleaner.maxAge.seconds. + </td> + </tr> + <tr> + <td>spark.history.fs.cleaner.maxAge.seconds</td> + <td>3600 * 24 * 7</td> + <td> + Job history files older than this many seconds will be deleted when the history cleaner runs. + Defaults to 3600 * 24 * 7 (1 week). + </td> + </tr> </table> Note that in all of these UIs, the tables are sortable by clicking their headers, |