diff options
| author | epahomov <pahomov.egor@gmail.com> | 2014-09-25 14:50:12 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@apache.org> | 2014-09-25 14:50:12 -0700 |
| commit | 9b56e249e09d8da20f703b9381c5c3c8a1a1d4a9 (patch) | |
| tree | e6cea0a4d71a8a3c04e0c1ddf8f2c5f59d2c39c0 | |
| parent | c3f2a8588e19aab814ac5cdd86575bb5558d5e46 (diff) | |
| download | spark-9b56e249e09d8da20f703b9381c5c3c8a1a1d4a9.tar.gz spark-9b56e249e09d8da20f703b9381c5c3c8a1a1d4a9.tar.bz2 spark-9b56e249e09d8da20f703b9381c5c3c8a1a1d4a9.zip | |
[SPARK-3690] Closing shuffle writers we swallow more important exception
Author: epahomov <pahomov.egor@gmail.com>
Closes #2537 from epahomov/SPARK-3690 and squashes the following commits:
a0b7de4 [epahomov] [SPARK-3690] Closing shuffle writers we swallow more important exception
| -rw-r--r-- | core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala | 9 |
1 files changed, 7 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala b/core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala index 381eff2147..a98ee11825 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala @@ -69,8 +69,13 @@ private[spark] class ShuffleMapTask( return writer.stop(success = true).get } catch { case e: Exception => - if (writer != null) { - writer.stop(success = false) + try { + if (writer != null) { + writer.stop(success = false) + } + } catch { + case e: Exception => + log.debug("Could not stop writer", e) } throw e } finally { |