diff options
author | Andrew Or <andrew@databricks.com> | 2015-03-03 15:09:57 -0800 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-03-03 15:09:57 -0800 |
commit | 6c20f35290e220e4a659a0222d62575ff959d703 (patch) | |
tree | 2bce225b3927c01c726f3a9e084e34900e211bb4 | |
parent | fe63e822918a01e1c1d741052b932e9944745fb6 (diff) | |
download | spark-6c20f35290e220e4a659a0222d62575ff959d703.tar.gz spark-6c20f35290e220e4a659a0222d62575ff959d703.tar.bz2 spark-6c20f35290e220e4a659a0222d62575ff959d703.zip |
[SPARK-6133] Make sc.stop() idempotent
Before we would get the following (benign) error if we called `sc.stop()` twice. This is because the listener bus would try to post the end event again even after it has already stopped. This happens occasionally when flaky tests fail, usually as a result of other sources of error. Either way we shouldn't be logging this error when it is not the cause of the failure.
```
ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerApplicationEnd(1425348445682)
```
Author: Andrew Or <andrew@databricks.com>
Closes #4871 from andrewor14/sc-stop and squashes the following commits:
a14afc5 [Andrew Or] Move code after code
915db16 [Andrew Or] Move code into code
-rw-r--r-- | core/src/main/scala/org/apache/spark/SparkContext.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala index e231e8369d..1a0bee4e3a 100644 --- a/core/src/main/scala/org/apache/spark/SparkContext.scala +++ b/core/src/main/scala/org/apache/spark/SparkContext.scala @@ -1392,10 +1392,10 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllocationCli /** Shut down the SparkContext. */ def stop() { SparkContext.SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized { - postApplicationEnd() - ui.foreach(_.stop()) if (!stopped) { stopped = true + postApplicationEnd() + ui.foreach(_.stop()) env.metricsSystem.report() metadataCleaner.cancel() cleaner.foreach(_.stop()) |