diff options
| author | Kay Ousterhout <kayousterhout@gmail.com> | 2015-05-28 22:09:49 -0700 |
|---|---|---|
| committer | Kay Ousterhout <kayousterhout@gmail.com> | 2015-05-28 22:09:49 -0700 |
| commit | 04ddcd4db7801abefa9c9effe5d88413b29d713b (patch) | |
| tree | 7571b3771b67ee011ec97295617096fae9439910 /core | |
| parent | 834e699524583a7ebfe9e83b3900ec503150deca (diff) | |
| download | spark-04ddcd4db7801abefa9c9effe5d88413b29d713b.tar.gz spark-04ddcd4db7801abefa9c9effe5d88413b29d713b.tar.bz2 spark-04ddcd4db7801abefa9c9effe5d88413b29d713b.zip | |
[SPARK-7932] Fix misleading scheduler delay visualization
The existing code rounds down to the nearest percent when computing the proportion
of a task's time that was spent on each phase of execution, and then computes
the scheduler delay proportion as 100 - sum(all other proportions). As a result,
a few extra percent can end up in the scheduler delay. This commit eliminates
the rounding so that the time visualizations correspond properly to the real times.
sarutak If you could take a look at this, that would be great! Not sure if there's a good
reason to round here that I missed.
cc shivaram
Author: Kay Ousterhout <kayousterhout@gmail.com>
Closes #6484 from kayousterhout/SPARK-7932 and squashes the following commits:
1723cc4 [Kay Ousterhout] [SPARK-7932] Fix misleading scheduler delay visualization
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala index 31e2e7fba9..b83a49f79c 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala @@ -527,7 +527,7 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") { minLaunchTime = launchTime.min(minLaunchTime) maxFinishTime = finishTime.max(maxFinishTime) - def toProportion(time: Long) = (time.toDouble / totalExecutionTime * 100).toLong + def toProportion(time: Long) = time.toDouble / totalExecutionTime * 100 val metricsOpt = taskUIData.taskMetrics val shuffleReadTime = |