diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-04-22 19:22:06 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-22 19:22:06 -0700 |
| commit | 995fdc96bcd2c540804401eaab009a777d7d7aa9 (patch) | |
| tree | c57cf54c46a3a33cce065682fb26422a42c0ca56 /conf | |
| parent | ea8cea82a02099bb66f1e77b757e4d96cc31d6e2 (diff) | |
| download | spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.tar.gz spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.tar.bz2 spark-995fdc96bcd2c540804401eaab009a777d7d7aa9.zip | |
Assorted clean-up for Spark-on-YARN.
In particular when the HADOOP_CONF_DIR is not not specified.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #488 from pwendell/hadoop-cleanup and squashes the following commits:
fe95f13 [Patrick Wendell] Changes based on Andrew's feeback
18d09c1 [Patrick Wendell] Review comments from Andrew
17929cc [Patrick Wendell] Assorted clean-up for Spark-on-YARN.
Diffstat (limited to 'conf')
| -rwxr-xr-x | conf/spark-env.sh.template | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/conf/spark-env.sh.template b/conf/spark-env.sh.template index 177a21cc03..f906be611a 100755 --- a/conf/spark-env.sh.template +++ b/conf/spark-env.sh.template @@ -5,6 +5,7 @@ # Options read when launching programs locally with # ./bin/run-example or ./bin/spark-submit +# - HADOOP_CONF_DIR, to point Spark towards Hadoop configuration files # - SPARK_LOCAL_IP, to set the IP address Spark binds to on this node # - SPARK_PUBLIC_DNS, to set the public dns name of the driver program # - SPARK_CLASSPATH, default classpath entries to append @@ -17,6 +18,7 @@ # - MESOS_NATIVE_LIBRARY, to point to your libmesos.so if you use Mesos # Options read in YARN client mode +# - HADOOP_CONF_DIR, to point Spark towards Hadoop configuration files # - SPARK_EXECUTOR_INSTANCES, Number of workers to start (Default: 2) # - SPARK_EXECUTOR_CORES, Number of cores for the workers (Default: 1). # - SPARK_EXECUTOR_MEMORY, Memory per Worker (e.g. 1000M, 2G) (Default: 1G) |