diff options
| author | Sean Owen <sowen@cloudera.com> | 2016-10-04 10:29:22 +0100 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-10-04 10:29:22 +0100 |
| commit | 8e8de0073d71bb00baeb24c612d7841b6274f652 (patch) | |
| tree | 83892704a5534bdf8d3d9419b1ba276cf00066cd /project | |
| parent | 126baa8d32bc0e7bf8b43f9efa84f2728f02347d (diff) | |
| download | spark-8e8de0073d71bb00baeb24c612d7841b6274f652.tar.gz spark-8e8de0073d71bb00baeb24c612d7841b6274f652.tar.bz2 spark-8e8de0073d71bb00baeb24c612d7841b6274f652.zip | |
[SPARK-17671][WEBUI] Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications
## What changes were proposed in this pull request?
Return Iterator of applications internally in history server, for consistency and performance. See https://github.com/apache/spark/pull/15248 for some back-story.
The code called by and calling HistoryServer.getApplicationList wants an Iterator, but this method materializes an Iterable, which potentially causes a performance problem. It's simpler too to make this internal method also pass through an Iterator.
## How was this patch tested?
Existing tests.
Author: Sean Owen <sowen@cloudera.com>
Closes #15321 from srowen/SPARK-17671.
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 7362041428..163e3f2fde 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -37,6 +37,8 @@ object MimaExcludes { // Exclude rules for 2.1.x lazy val v21excludes = v20excludes ++ { Seq( + // [SPARK-17671] Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications + ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.deploy.history.HistoryServer.getApplicationList"), // [SPARK-14743] Improve delegation token handling in secure cluster ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkHadoopUtil.getTimeFromNowToRenewal"), // [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter |