diff options
| author | Alex Bozarth <ajbozart@us.ibm.com> | 2016-08-24 14:39:41 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-08-24 14:39:41 -0500 |
| commit | 891ac2b914fb6f90a62c6fbc0a3960a89d1c1d92 (patch) | |
| tree | 19d1eea2303c7b29c310047d40e04a1b5f694cc7 /python | |
| parent | 40b30fcf453169534cb53d01cd22236210b13005 (diff) | |
| download | spark-891ac2b914fb6f90a62c6fbc0a3960a89d1c1d92.tar.gz spark-891ac2b914fb6f90a62c6fbc0a3960a89d1c1d92.tar.bz2 spark-891ac2b914fb6f90a62c6fbc0a3960a89d1c1d92.zip | |
[SPARK-15083][WEB UI] History Server can OOM due to unlimited TaskUIData
## What changes were proposed in this pull request?
Based on #12990 by tankkyo
Since the History Server currently loads all application's data it can OOM if too many applications have a significant task count. `spark.ui.trimTasks` (default: false) can be set to true to trim tasks by `spark.ui.retainedTasks` (default: 10000)
(This is a "quick fix" to help those running into the problem until a update of how the history server loads app data can be done)
## How was this patch tested?
Manual testing and dev/run-tests
Author: Alex Bozarth <ajbozart@us.ibm.com>
Closes #14673 from ajbozarth/spark15083.
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions