diff options
| author | Eric Liang <ekl@databricks.com> | 2017-04-20 09:55:10 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2017-04-20 09:55:10 -0700 |
| commit | b2ebadfd55283348b8a8b37e28075fca0798228a (patch) | |
| tree | ec79f4bdfa27b9241a0715dcd796677ac66ca73b /core/src/main/scala | |
| parent | c5a31d160f47ba51bb9f8a4f3141851034640fc7 (diff) | |
| download | spark-b2ebadfd55283348b8a8b37e28075fca0798228a.tar.gz spark-b2ebadfd55283348b8a8b37e28075fca0798228a.tar.bz2 spark-b2ebadfd55283348b8a8b37e28075fca0798228a.zip | |
[SPARK-20358][CORE] Executors failing stage on interrupted exception thrown by cancelled tasks
## What changes were proposed in this pull request?
This was a regression introduced by my earlier PR here: https://github.com/apache/spark/pull/17531
It turns out NonFatal() does not in fact catch InterruptedException.
## How was this patch tested?
Extended cancellation unit test coverage. The first test fails before this patch.
cc JoshRosen mridulm
Author: Eric Liang <ekl@databricks.com>
Closes #17659 from ericl/spark-20358.
Diffstat (limited to 'core/src/main/scala')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/executor/Executor.scala | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/executor/Executor.scala b/core/src/main/scala/org/apache/spark/executor/Executor.scala index 83469c5ff0..18f04391d6 100644 --- a/core/src/main/scala/org/apache/spark/executor/Executor.scala +++ b/core/src/main/scala/org/apache/spark/executor/Executor.scala @@ -432,7 +432,8 @@ private[spark] class Executor( setTaskFinishedAndClearInterruptStatus() execBackend.statusUpdate(taskId, TaskState.KILLED, ser.serialize(TaskKilled(t.reason))) - case NonFatal(_) if task != null && task.reasonIfKilled.isDefined => + case _: InterruptedException | NonFatal(_) if + task != null && task.reasonIfKilled.isDefined => val killReason = task.reasonIfKilled.getOrElse("unknown reason") logInfo(s"Executor interrupted and killed $taskName (TID $taskId), reason: $killReason") setTaskFinishedAndClearInterruptStatus() |