diff options
| author | Eric Liang <ekl@databricks.com> | 2017-04-19 19:53:40 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2017-04-19 19:53:40 -0700 |
| commit | dd6d55d5de970662eccf024e5eae4e6821373d35 (patch) | |
| tree | 881763a10b67eb078109566b0827c3168312e0cb | |
| parent | 39e303a8b6db642c26dbc26ba92e87680f50e4da (diff) | |
| download | spark-dd6d55d5de970662eccf024e5eae4e6821373d35.tar.gz spark-dd6d55d5de970662eccf024e5eae4e6821373d35.tar.bz2 spark-dd6d55d5de970662eccf024e5eae4e6821373d35.zip | |
[SPARK-20398][SQL] range() operator should include cancellation reason when killed
## What changes were proposed in this pull request?
https://issues.apache.org/jira/browse/SPARK-19820 adds a reason field for why tasks were killed. However, for backwards compatibility it left the old TaskKilledException constructor which defaults to "unknown reason".
The range() operator should use the constructor that fills in the reason rather than dropping it on task kill.
## How was this patch tested?
Existing tests, and I tested this manually.
Author: Eric Liang <ekl@databricks.com>
Closes #17692 from ericl/fix-kill-reason-in-range.
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala index 44278e37c5..233a105f4d 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala @@ -463,9 +463,7 @@ case class RangeExec(range: org.apache.spark.sql.catalyst.plans.logical.Range) | $number = $batchEnd; | } | - | if ($taskContext.isInterrupted()) { - | throw new TaskKilledException(); - | } + | $taskContext.killTaskIfInterrupted(); | | long $nextBatchTodo; | if ($numElementsTodo > ${batchSize}L) { |