diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-04-22 19:22:06 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-22 19:22:06 -0700 |
| commit | 995fdc96bcd2c540804401eaab009a777d7d7aa9 (patch) | |
| tree | c57cf54c46a3a33cce065682fb26422a42c0ca56 /yarn | |
| parent | ea8cea82a02099bb66f1e77b757e4d96cc31d6e2 (diff) | |
| download | spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.tar.gz spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.tar.bz2 spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.zip | |
Assorted clean-up for Spark-on-YARN.
In particular when the HADOOP_CONF_DIR is not not specified.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #488 from pwendell/hadoop-cleanup and squashes the following commits:
fe95f13 [Patrick Wendell] Changes based on Andrew's feeback
18d09c1 [Patrick Wendell] Review comments from Andrew
17929cc [Patrick Wendell] Assorted clean-up for Spark-on-YARN.
Diffstat (limited to 'yarn')
| -rw-r--r-- | yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala b/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala index d264ecbeaf..b403292d9c 100644 --- a/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala +++ b/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala @@ -375,9 +375,11 @@ object ClientBase { val classpathEntries = Option(conf.getStrings( YarnConfiguration.YARN_APPLICATION_CLASSPATH)).getOrElse( getDefaultYarnApplicationClasspath()) - for (c <- classpathEntries) { - YarnSparkHadoopUtil.addToEnvironment(env, Environment.CLASSPATH.name, c.trim, - File.pathSeparator) + if (classpathEntries != null) { + for (c <- classpathEntries) { + YarnSparkHadoopUtil.addToEnvironment(env, Environment.CLASSPATH.name, c.trim, + File.pathSeparator) + } } val mrClasspathEntries = Option(conf.getStrings( |