diff options
| author | shiyun.wxm <shiyun.wxm@taobao.com> | 2014-03-19 01:42:34 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-03-19 01:42:34 -0700 |
| commit | d55ec86de2e96f7dc9d1dd107daa35c3823791ec (patch) | |
| tree | 79dc6fd20ad131eeda914c393e9e2615682de2cf | |
| parent | a18ea00f3af0fa4c6b2c59933e22b6c9f0f636c8 (diff) | |
| download | spark-d55ec86de2e96f7dc9d1dd107daa35c3823791ec.tar.gz spark-d55ec86de2e96f7dc9d1dd107daa35c3823791ec.tar.bz2 spark-d55ec86de2e96f7dc9d1dd107daa35c3823791ec.zip | |
bugfix: Wrong "Duration" in "Active Stages" in stages page
If a stage which has completed once loss parts of data, it will be resubmitted. At this time, it appears that stage.completionTime > stage.submissionTime.
Author: shiyun.wxm <shiyun.wxm@taobao.com>
Closes #170 from BlackNiuza/duration_problem and squashes the following commits:
a86d261 [shiyun.wxm] tow space indent
c0d7b24 [shiyun.wxm] change the style
3b072e1 [shiyun.wxm] fix scala style
f20701e [shiyun.wxm] bugfix: "Duration" in "Active Stages" in stages page
| -rw-r--r-- | core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala index c5fd3ae16d..99828487f1 100644 --- a/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala +++ b/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala @@ -106,7 +106,9 @@ private[spark] class StageTable(val stages: Seq[StageInfo], val parent: JobProgr val description = listener.stageIdToDescription.get(s.stageId) .map(d => <div><em>{d}</em></div><div>{nameLink}</div>).getOrElse(nameLink) val finishTime = s.completionTime.getOrElse(System.currentTimeMillis()) - val duration = s.submissionTime.map(t => finishTime - t) + val duration = s.submissionTime.map { t => + if (finishTime > t) finishTime - t else System.currentTimeMillis - t + } <tr> <td>{s.stageId}</td> |