diff options
| author | Jacek Lewandowski <lewandowski.jacek@gmail.com> | 2015-01-25 15:20:50 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-01-25 15:20:50 -0800 |
| commit | e82e9601ce68978335d97e08f3cb690313cc1064 (patch) | |
| tree | 0a1d5c86bc1c826943db5c5ff5da69aec7c66796 /bin | |
| parent | 7652809be1ae02b4d66dcdf77b448f6d08b7d902 (diff) | |
| download | spark-e82e9601ce68978335d97e08f3cb690313cc1064.tar.gz spark-e82e9601ce68978335d97e08f3cb690313cc1064.tar.bz2 spark-e82e9601ce68978335d97e08f3cb690313cc1064.zip | |
SPARK-5382: Use SPARK_CONF_DIR in spark-class and spark-submit, spark-su...
...bmit2.cmd if it is defined
Author: Jacek Lewandowski <lewandowski.jacek@gmail.com>
Closes #4177 from jacek-lewandowski/SPARK-5382-1.2 and squashes the following commits:
41cef25 [Jacek Lewandowski] SPARK-5382: Use SPARK_CONF_DIR in spark-class and spark-submit, spark-submit2.cmd if it is defined
Diffstat (limited to 'bin')
| -rwxr-xr-x | bin/spark-submit | 4 | ||||
| -rw-r--r-- | bin/spark-submit2.cmd | 7 |
2 files changed, 9 insertions, 2 deletions
diff --git a/bin/spark-submit b/bin/spark-submit index aefd38a0a2..216b92e411 100755 --- a/bin/spark-submit +++ b/bin/spark-submit @@ -20,6 +20,8 @@ # NOTE: Any changes in this file must be reflected in SparkSubmitDriverBootstrapper.scala! export SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)" +export SPARK_CONF_DIR="${SPARK_CONF_DIR:-"$SPARK_HOME/conf"}" + ORIG_ARGS=("$@") # Set COLUMNS for progress bar @@ -44,7 +46,7 @@ while (($#)); do shift done -DEFAULT_PROPERTIES_FILE="$SPARK_HOME/conf/spark-defaults.conf" +DEFAULT_PROPERTIES_FILE="$SPARK_CONF_DIR/spark-defaults.conf" if [ "$MASTER" == "yarn-cluster" ]; then SPARK_SUBMIT_DEPLOY_MODE=cluster fi diff --git a/bin/spark-submit2.cmd b/bin/spark-submit2.cmd index daf0284db9..4581264b58 100644 --- a/bin/spark-submit2.cmd +++ b/bin/spark-submit2.cmd @@ -24,13 +24,18 @@ set ORIG_ARGS=%* rem Reset the values of all variables used set SPARK_SUBMIT_DEPLOY_MODE=client -set SPARK_SUBMIT_PROPERTIES_FILE=%SPARK_HOME%\conf\spark-defaults.conf set SPARK_SUBMIT_DRIVER_MEMORY= set SPARK_SUBMIT_LIBRARY_PATH= set SPARK_SUBMIT_CLASSPATH= set SPARK_SUBMIT_OPTS= set SPARK_SUBMIT_BOOTSTRAP_DRIVER= +if not "x%SPARK_CONF_DIR%"=="x" ( + set SPARK_SUBMIT_PROPERTIES_FILE=%SPARK_CONF_DIR%\spark-defaults.conf +) else ( + set SPARK_SUBMIT_PROPERTIES_FILE=%SPARK_HOME%\conf\spark-defaults.conf +) + :loop if [%1] == [] goto continue if [%1] == [--deploy-mode] ( |