diff options
| author | Kousuke Saruta <sarutak@oss.nttdata.co.jp> | 2016-09-02 10:26:43 +0100 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-09-02 10:26:43 +0100 |
| commit | 7ee24dac8e779f6a9bf45371fdc2be83fb679cb2 (patch) | |
| tree | 3f8115c8807470ccb29c1c795567349ce193a166 /core/src | |
| parent | a3097e2b314b1cd7446c6106a54fe4b49871c26d (diff) | |
| download | spark-7ee24dac8e779f6a9bf45371fdc2be83fb679cb2.tar.gz spark-7ee24dac8e779f6a9bf45371fdc2be83fb679cb2.tar.bz2 spark-7ee24dac8e779f6a9bf45371fdc2be83fb679cb2.zip | |
[SPARK-17352][WEBUI] Executor computing time can be negative-number because of calculation error
## What changes were proposed in this pull request?
In StagePage, executor-computing-time is calculated but calculation error can occur potentially because it's calculated by subtraction of floating numbers.
Following capture is an example.
<img width="949" alt="capture-timeline" src="https://cloud.githubusercontent.com/assets/4736016/18152359/43f07a28-7030-11e6-8cbd-8e73bf4c4c67.png">
## How was this patch tested?
Manual tests.
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Closes #14908 from sarutak/SPARK-17352.
Diffstat (limited to 'core/src')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala index a266164587..de787f2577 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala @@ -651,9 +651,9 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") { } val executorComputingTime = executorRunTime - shuffleReadTime - shuffleWriteTime val executorComputingTimeProportion = - (100 - schedulerDelayProportion - shuffleReadTimeProportion - + math.max(100 - schedulerDelayProportion - shuffleReadTimeProportion - shuffleWriteTimeProportion - serializationTimeProportion - - deserializationTimeProportion - gettingResultTimeProportion) + deserializationTimeProportion - gettingResultTimeProportion, 0) val schedulerDelayProportionPos = 0 val deserializationTimeProportionPos = |