diff options
| author | WangTaoTheTonic <barneystinson@aliyun.com> | 2015-01-08 11:45:42 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-01-08 11:45:42 -0800 |
| commit | 0760787da885187b0c6dcd5c28753f0ab014d5ed (patch) | |
| tree | 5335ab4101c324d14719eb244b687e5697918442 /bin/spark-submit | |
| parent | 0a597276dbfd921665a9f720d96e119058186aa4 (diff) | |
| download | spark-0760787da885187b0c6dcd5c28753f0ab014d5ed.tar.gz spark-0760787da885187b0c6dcd5c28753f0ab014d5ed.tar.bz2 spark-0760787da885187b0c6dcd5c28753f0ab014d5ed.zip | |
[SPARK-5130][Deploy]Take yarn-cluster as cluster mode in spark-submit
https://issues.apache.org/jira/browse/SPARK-5130
Author: WangTaoTheTonic <barneystinson@aliyun.com>
Closes #3929 from WangTaoTheTonic/SPARK-5130 and squashes the following commits:
c490648 [WangTaoTheTonic] take yarn-cluster as cluster mode in spark-submit
Diffstat (limited to 'bin/spark-submit')
| -rwxr-xr-x | bin/spark-submit | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/bin/spark-submit b/bin/spark-submit index f92d90c3a6..aefd38a0a2 100755 --- a/bin/spark-submit +++ b/bin/spark-submit @@ -38,11 +38,16 @@ while (($#)); do export SPARK_SUBMIT_CLASSPATH=$2 elif [ "$1" = "--driver-java-options" ]; then export SPARK_SUBMIT_OPTS=$2 + elif [ "$1" = "--master" ]; then + export MASTER=$2 fi shift done DEFAULT_PROPERTIES_FILE="$SPARK_HOME/conf/spark-defaults.conf" +if [ "$MASTER" == "yarn-cluster" ]; then + SPARK_SUBMIT_DEPLOY_MODE=cluster +fi export SPARK_SUBMIT_DEPLOY_MODE=${SPARK_SUBMIT_DEPLOY_MODE:-"client"} export SPARK_SUBMIT_PROPERTIES_FILE=${SPARK_SUBMIT_PROPERTIES_FILE:-"$DEFAULT_PROPERTIES_FILE"} |