diff options
author | jerryshao <sshao@hortonworks.com> | 2016-03-18 12:39:49 -0700 |
---|---|---|
committer | Marcelo Vanzin <vanzin@cloudera.com> | 2016-03-18 12:39:49 -0700 |
commit | 3537782168aa9278ac4add1a25afac0ec6e17085 (patch) | |
tree | cb5fcec81608a012b6027e2b3ad25313371f50fc /core | |
parent | 9c23c818ca0175c8f2a4a66eac261ec251d27c97 (diff) | |
download | spark-3537782168aa9278ac4add1a25afac0ec6e17085.tar.gz spark-3537782168aa9278ac4add1a25afac0ec6e17085.tar.bz2 spark-3537782168aa9278ac4add1a25afac0ec6e17085.zip |
[SPARK-13885][YARN] Fix attempt id regression for Spark running on Yarn
## What changes were proposed in this pull request?
This regression is introduced in #9182, previously attempt id is simply as counter "1" or "2". With the change of #9182, it is changed to full name as "appattemtp-xxx-00001", this will affect all the parts which uses this attempt id, like event log file name, history server app url link. So here change it back to the counter to keep consistent with previous code.
Also revert back this patch #11518, this patch fix the url link of history log according to the new way of attempt id, since here we change back to the previous way, so this patch is not necessary, here to revert it.
Also clean "spark.yarn.app.id" and "spark.yarn.app.attemptId", since it is useless now.
## How was this patch tested?
Test it with unit test and manually test different scenario:
1. application running in yarn-client mode.
2. application running in yarn-cluster mode.
3. application running in yarn-cluster mode with multiple attempts.
Checked both the event log file name and url link.
CC vanzin tgravescs , please help to review, thanks a lot.
Author: jerryshao <sshao@hortonworks.com>
Closes #11721 from jerryshao/SPARK-13885.
Diffstat (limited to 'core')
-rw-r--r-- | core/src/main/resources/org/apache/spark/ui/static/historypage-template.html | 4 | ||||
-rw-r--r-- | core/src/main/resources/org/apache/spark/ui/static/historypage.js | 19 |
2 files changed, 4 insertions, 19 deletions
diff --git a/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html b/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html index 5a7a252231..a2b3826dd3 100644 --- a/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html +++ b/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html @@ -64,10 +64,10 @@ <tbody> {{#applications}} <tr> - <td class="rowGroupColumn"><span title="{{id}}"><a href="{{url}}">{{id}}</a></span></td> + <td class="rowGroupColumn"><span title="{{id}}"><a href="/history/{{id}}/{{num}}/jobs/">{{id}}</a></span></td> <td class="rowGroupColumn">{{name}}</td> {{#attempts}} - <td class="attemptIDSpan"><a href="history/{{id}}/{{attemptId}}/">{{attemptId}}</a></td> + <td class="attemptIDSpan"><a href="/history/{{id}}/{{attemptId}}/jobs/">{{attemptId}}</a></td> <td>{{startTime}}</td> <td>{{endTime}}</td> <td><span title="{{duration}}" class="durationClass">{{duration}}</span></td> diff --git a/core/src/main/resources/org/apache/spark/ui/static/historypage.js b/core/src/main/resources/org/apache/spark/ui/static/historypage.js index 6096513154..ef89a9a86f 100644 --- a/core/src/main/resources/org/apache/spark/ui/static/historypage.js +++ b/core/src/main/resources/org/apache/spark/ui/static/historypage.js @@ -123,28 +123,13 @@ $(document).ready(function() { if (app["attempts"].length > 1) { hasMultipleAttempts = true; } - - var maxAttemptId = null + var num = app["attempts"].length; for (j in app["attempts"]) { var attempt = app["attempts"][j]; - if (attempt['attemptId'] != null) { - if (maxAttemptId == null || attempt['attemptId'] > maxAttemptId) { - maxAttemptId = attempt['attemptId'] - } - } - attempt["startTime"] = formatDate(attempt["startTime"]); attempt["endTime"] = formatDate(attempt["endTime"]); attempt["lastUpdated"] = formatDate(attempt["lastUpdated"]); - - var url = null - if (maxAttemptId == null) { - url = "history/" + id + "/" - } else { - url = "history/" + id + "/" + maxAttemptId + "/" - } - - var app_clone = {"id" : id, "name" : name, "url" : url, "attempts" : [attempt]}; + var app_clone = {"id" : id, "name" : name, "num" : num, "attempts" : [attempt]}; array.push(app_clone); } } |