diff options
author | Reynold Xin <rxin@apache.org> | 2014-08-20 15:37:27 -0700 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-08-20 15:37:27 -0700 |
commit | fb60bec34e0b20ae95b4b865a79744916e0a5737 (patch) | |
tree | a9f9bb8d0d0e749b72f7b38c52b6a5e0f7a4985f /sbin | |
parent | b3ec51bfd795772ff96d18228e979a52ebc82ec4 (diff) | |
download | spark-fb60bec34e0b20ae95b4b865a79744916e0a5737.tar.gz spark-fb60bec34e0b20ae95b4b865a79744916e0a5737.tar.bz2 spark-fb60bec34e0b20ae95b4b865a79744916e0a5737.zip |
[SPARK-2298] Encode stage attempt in SparkListener & UI.
Simple way to reproduce this in the UI:
```scala
val f = new java.io.File("/tmp/test")
f.delete()
sc.parallelize(1 to 2, 2).map(x => (x,x )).repartition(3).mapPartitionsWithContext { case (context, iter) =>
if (context.partitionId == 0) {
val f = new java.io.File("/tmp/test")
if (!f.exists) {
f.mkdir()
System.exit(0);
}
}
iter
}.count()
```
Author: Reynold Xin <rxin@apache.org>
Closes #1545 from rxin/stage-attempt and squashes the following commits:
3ee1d2a [Reynold Xin] - Rename attempt to retry in UI. - Properly report stage failure in FetchFailed.
40a6bd5 [Reynold Xin] Updated test suites.
c414c36 [Reynold Xin] Fixed the hanging in JobCancellationSuite.
b3e2eed [Reynold Xin] Oops previous code didn't compile.
0f36075 [Reynold Xin] Mark unknown stage attempt with id -1 and drop that in JobProgressListener.
6c08b07 [Reynold Xin] Addressed code review feedback.
4e5faa2 [Reynold Xin] [SPARK-2298] Encode stage attempt in SparkListener & UI.
Diffstat (limited to 'sbin')
0 files changed, 0 insertions, 0 deletions