diff options
Diffstat (limited to 'docs/submitting-applications.md')
-rw-r--r-- | docs/submitting-applications.md | 25 |
1 files changed, 15 insertions, 10 deletions
diff --git a/docs/submitting-applications.md b/docs/submitting-applications.md index 915be0f479..ac2a14eb56 100644 --- a/docs/submitting-applications.md +++ b/docs/submitting-applications.md @@ -103,7 +103,8 @@ run it with `--help`. Here are a few examples of common options: export HADOOP_CONF_DIR=XXX ./bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ - --master yarn-cluster \ # can also be yarn-client for client mode + --master yarn \ + --deploy-mode cluster \ # can be client for client mode --executor-memory 20G \ --num-executors 50 \ /path/to/examples.jar \ @@ -122,21 +123,25 @@ The master URL passed to Spark can be in one of the following formats: <table class="table"> <tr><th>Master URL</th><th>Meaning</th></tr> -<tr><td> local </td><td> Run Spark locally with one worker thread (i.e. no parallelism at all). </td></tr> -<tr><td> local[K] </td><td> Run Spark locally with K worker threads (ideally, set this to the number of cores on your machine). </td></tr> -<tr><td> local[*] </td><td> Run Spark locally with as many worker threads as logical cores on your machine.</td></tr> -<tr><td> spark://HOST:PORT </td><td> Connect to the given <a href="spark-standalone.html">Spark standalone +<tr><td> <code>local</code> </td><td> Run Spark locally with one worker thread (i.e. no parallelism at all). </td></tr> +<tr><td> <code>local[K]</code> </td><td> Run Spark locally with K worker threads (ideally, set this to the number of cores on your machine). </td></tr> +<tr><td> <code>local[*]</code> </td><td> Run Spark locally with as many worker threads as logical cores on your machine.</td></tr> +<tr><td> <code>spark://HOST:PORT</code> </td><td> Connect to the given <a href="spark-standalone.html">Spark standalone cluster</a> master. The port must be whichever one your master is configured to use, which is 7077 by default. </td></tr> -<tr><td> mesos://HOST:PORT </td><td> Connect to the given <a href="running-on-mesos.html">Mesos</a> cluster. +<tr><td> <code>mesos://HOST:PORT</code> </td><td> Connect to the given <a href="running-on-mesos.html">Mesos</a> cluster. The port must be whichever one your is configured to use, which is 5050 by default. Or, for a Mesos cluster using ZooKeeper, use <code>mesos://zk://...</code>. </td></tr> -<tr><td> yarn-client </td><td> Connect to a <a href="running-on-yarn.html"> YARN </a> cluster in -client mode. The cluster location will be found based on the HADOOP_CONF_DIR or YARN_CONF_DIR variable. +<tr><td> <code>yarn</code> </td><td> Connect to a <a href="running-on-yarn.html"> YARN </a> cluster in + <code>client</code> or <code>cluster</code> mode depending on the value of <code>--deploy-mode</code>. + The cluster location will be found based on the <code>HADOOP_CONF_DIR</code> or <code>YARN_CONF_DIR</code> variable. </td></tr> -<tr><td> yarn-cluster </td><td> Connect to a <a href="running-on-yarn.html"> YARN </a> cluster in -cluster mode. The cluster location will be found based on the HADOOP_CONF_DIR or YARN_CONF_DIR variable. +<tr><td> <code>yarn-client</code> </td><td> Equivalent to <code>yarn</code> with <code>--deploy-mode client</code>, + which is preferred to `yarn-client` +</td></tr> +<tr><td> <code>yarn-cluster</code> </td><td> Equivalent to <code>yarn</code> with <code>--deploy-mode cluster</code>, + which is preferred to `yarn-cluster` </td></tr> </table> |