diff options
| author | Rajesh Balamohan <rbalamohan@apache.org> | 2016-08-10 15:30:22 -0700 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2016-08-10 15:30:52 -0700 |
| commit | bd2c12fb4994785d5becce541aee9ba73fef1c4c (patch) | |
| tree | 0baac3281868c4da4ca66ef473a901c68b30c519 /docs/running-on-yarn.md | |
| parent | bf5cb8af4a649e0c7ac565891427484eab9ee5d9 (diff) | |
| download | spark-bd2c12fb4994785d5becce541aee9ba73fef1c4c.tar.gz spark-bd2c12fb4994785d5becce541aee9ba73fef1c4c.tar.bz2 spark-bd2c12fb4994785d5becce541aee9ba73fef1c4c.zip | |
[SPARK-12920][CORE] Honor "spark.ui.retainedStages" to reduce mem-pressure
When large number of jobs are run concurrently with Spark thrift server, thrift server starts running at high CPU due to GC pressure. Job UI retention causes memory pressure with large jobs. https://issues.apache.org/jira/secure/attachment/12783302/SPARK-12920.profiler_job_progress_listner.png has the profiler snapshot. This PR honors `spark.ui.retainedStages` strictly to reduce memory pressure.
Manual and unit tests
Author: Rajesh Balamohan <rbalamohan@apache.org>
Closes #10846 from rajeshbalamohan/SPARK-12920.
Diffstat (limited to 'docs/running-on-yarn.md')
0 files changed, 0 insertions, 0 deletions