diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-08-11 23:23:17 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-08-11 23:23:17 -0700 |
| commit | 4e3f4b934f74e8c7c06f4940d6381343f9fd4918 (patch) | |
| tree | d70475fb9fe79e97973aca68d00debea4d594e65 /core | |
| parent | a807fcbe50b2ce18751d80d39e9d21842f7da32a (diff) | |
| download | spark-4e3f4b934f74e8c7c06f4940d6381343f9fd4918.tar.gz spark-4e3f4b934f74e8c7c06f4940d6381343f9fd4918.tar.bz2 spark-4e3f4b934f74e8c7c06f4940d6381343f9fd4918.zip | |
[SPARK-9829] [WEBUI] Display the update value for peak execution memory
The peak execution memory is not correct because it shows the sum of finished tasks' values when a task finishes.
This PR fixes it by using the update value rather than the accumulator value.
Author: zsxwing <zsxwing@gmail.com>
Closes #8121 from zsxwing/SPARK-9829.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala index 0c94204df6..fb4556b836 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala @@ -860,7 +860,7 @@ private[ui] class TaskDataSource( } val peakExecutionMemoryUsed = taskInternalAccumulables .find { acc => acc.name == InternalAccumulator.PEAK_EXECUTION_MEMORY } - .map { acc => acc.value.toLong } + .map { acc => acc.update.getOrElse("0").toLong } .getOrElse(0L) val maybeInput = metrics.flatMap(_.inputMetrics) |