diff options
| author | Lianhui Wang <lianhuiwang09@gmail.com> | 2015-07-16 19:31:14 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-07-16 19:31:45 -0700 |
| commit | 49351c7f597c67950cc65e5014a89fad31b9a6f7 (patch) | |
| tree | 887471f1d7c0f9efd3e7e9f864c9e0c7ff6fda1c /yarn | |
| parent | 57e9b13bf99ab82896114d47c62fbee648c9be7d (diff) | |
| download | spark-49351c7f597c67950cc65e5014a89fad31b9a6f7.tar.gz spark-49351c7f597c67950cc65e5014a89fad31b9a6f7.tar.bz2 spark-49351c7f597c67950cc65e5014a89fad31b9a6f7.zip | |
[SPARK-8646] PySpark does not run on YARN if master not provided in command line
andrewor14 davies vanzin can you take a look at this? thanks
Author: Lianhui Wang <lianhuiwang09@gmail.com>
Closes #7438 from lianhuiwang/SPARK-8646 and squashes the following commits:
cb3f12d [Lianhui Wang] add whitespace
6d874a6 [Lianhui Wang] support pyspark for yarn-client
Diffstat (limited to 'yarn')
| -rw-r--r-- | yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala index f86b6d1e5d..b74ea9a10a 100644 --- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala +++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala @@ -616,7 +616,7 @@ private[spark] class Client( val appId = newAppResponse.getApplicationId val appStagingDir = getAppStagingDir(appId) val pySparkArchives = - if (sys.props.getOrElse("spark.yarn.isPython", "false").toBoolean) { + if (sparkConf.getBoolean("spark.yarn.isPython", false)) { findPySparkArchives() } else { Nil |