diff options
| author | Sean Owen <sowen@cloudera.com> | 2015-02-26 14:08:56 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-02-26 14:08:56 -0800 |
| commit | 3fb53c0298761ba227890525ae79ce4ec6300deb (patch) | |
| tree | a76b236d50003026c531f38ac0918eb10ae80d85 /core | |
| parent | 5f3238b3b0157091d28803aa3b1d248dfa6cdc59 (diff) | |
| download | spark-3fb53c0298761ba227890525ae79ce4ec6300deb.tar.gz spark-3fb53c0298761ba227890525ae79ce4ec6300deb.tar.bz2 spark-3fb53c0298761ba227890525ae79ce4ec6300deb.zip | |
SPARK-4300 [CORE] Race condition during SparkWorker shutdown
Close appender saving stdout/stderr before destroying process to avoid exception on reading closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)
CC tdas since I think you wrote this method.
Author: Sean Owen <sowen@cloudera.com>
Closes #4787 from srowen/SPARK-4300 and squashes the following commits:
e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying process to avoid exception on reading closed input stream
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala index 6653aca0a0..066d46c447 100644 --- a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala +++ b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala @@ -85,14 +85,13 @@ private[spark] class ExecutorRunner( var exitCode: Option[Int] = None if (process != null) { logInfo("Killing process!") - process.destroy() - process.waitFor() if (stdoutAppender != null) { stdoutAppender.stop() } if (stderrAppender != null) { stderrAppender.stop() } + process.destroy() exitCode = Some(process.waitFor()) } worker ! ExecutorStateChanged(appId, execId, state, message, exitCode) |