diff options
| author | Yuming Wang <wgyumg@gmail.com> | 2017-01-23 11:02:22 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2017-01-23 11:02:22 +0000 |
| commit | c99492141b1ddddb8edb6841a6e83748e5ba9bba (patch) | |
| tree | 494ac6360097d0da4fe12a96a73d274dd2ce1800 /docs | |
| parent | c4a6519c44f29950ef3d706a4f79e006ec8bc6b5 (diff) | |
| download | spark-c99492141b1ddddb8edb6841a6e83748e5ba9bba.tar.gz spark-c99492141b1ddddb8edb6841a6e83748e5ba9bba.tar.bz2 spark-c99492141b1ddddb8edb6841a6e83748e5ba9bba.zip | |
[SPARK-19146][CORE] Drop more elements when stageData.taskData.size > retainedTasks
## What changes were proposed in this pull request?
Drop more elements when `stageData.taskData.size > retainedTasks` to reduce the number of times on call drop function.
## How was this patch tested?
Jenkins
Author: Yuming Wang <wgyumg@gmail.com>
Closes #16527 from wangyum/SPARK-19146.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 12 |
1 files changed, 6 insertions, 6 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 7a11a983d5..a6b1f15fda 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -684,24 +684,24 @@ Apart from these, the following properties are also available, and may be useful <td><code>spark.ui.retainedJobs</code></td> <td>1000</td> <td> - How many jobs the Spark UI and status APIs remember before garbage - collecting. + How many jobs the Spark UI and status APIs remember before garbage collecting. + This is a target maximum, and fewer elements may be retained in some circumstances. </td> </tr> <tr> <td><code>spark.ui.retainedStages</code></td> <td>1000</td> <td> - How many stages the Spark UI and status APIs remember before garbage - collecting. + How many stages the Spark UI and status APIs remember before garbage collecting. + This is a target maximum, and fewer elements may be retained in some circumstances. </td> </tr> <tr> <td><code>spark.ui.retainedTasks</code></td> <td>100000</td> <td> - How many tasks the Spark UI and status APIs remember before garbage - collecting. + How many tasks the Spark UI and status APIs remember before garbage collecting. + This is a target maximum, and fewer elements may be retained in some circumstances. </td> </tr> <tr> |