diff options
| author | Reynold Xin <rxin@databricks.com> | 2016-02-26 12:49:16 -0800 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-02-26 12:49:16 -0800 |
| commit | 391755dc6ed2e156b8df8a530ac8df6ed7ba7f8a (patch) | |
| tree | 7ebb73ea7e90c5a4c180f493c8d8bdaffbf2d8f2 /project | |
| parent | 0598a2b81d1426dd2cf9e6fc32cef345364d18c6 (diff) | |
| download | spark-391755dc6ed2e156b8df8a530ac8df6ed7ba7f8a.tar.gz spark-391755dc6ed2e156b8df8a530ac8df6ed7ba7f8a.tar.bz2 spark-391755dc6ed2e156b8df8a530ac8df6ed7ba7f8a.zip | |
[SPARK-13465] Add a task failure listener to TaskContext
## What changes were proposed in this pull request?
TaskContext supports task completion callback, which gets called regardless of task failures. However, there is no way for the listener to know if there is an error. This patch adds a new listener that gets called when a task fails.
## How was the this patch tested?
New unit test case and integration test case covering the code path
Author: Reynold Xin <rxin@databricks.com>
Closes #11340 from rxin/SPARK-13465.
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 14e3c90f1b..165280a1b2 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -271,7 +271,9 @@ object MimaExcludes { ) ++ Seq( // SPARK-13220 Deprecate yarn-client and yarn-cluster mode ProblemFilters.exclude[MissingMethodProblem]( - "org.apache.spark.SparkContext.org$apache$spark$SparkContext$$createTaskScheduler") + "org.apache.spark.SparkContext.org$apache$spark$SparkContext$$createTaskScheduler"), + // SPARK-13465 TaskContext. + ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.TaskContext.addTaskFailureListener") ) ++ Seq ( // SPARK-7729 Executor which has been killed should also be displayed on Executor Tab ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.status.api.v1.ExecutorSummary.this") |