diff options
| author | Timothy Chen <tnachen@gmail.com> | 2016-02-01 12:45:02 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-02-01 12:45:02 -0800 |
| commit | 51b03b71ffc390e67b32936efba61e614a8b0d86 (patch) | |
| tree | e4f30c0054c46d8a52a8d7b1dd97045de5c53023 /docs/running-on-mesos.md | |
| parent | 711ce048a285403241bbc9eaabffc1314162e89c (diff) | |
| download | spark-51b03b71ffc390e67b32936efba61e614a8b0d86.tar.gz spark-51b03b71ffc390e67b32936efba61e614a8b0d86.tar.bz2 spark-51b03b71ffc390e67b32936efba61e614a8b0d86.zip | |
[SPARK-12463][SPARK-12464][SPARK-12465][SPARK-10647][MESOS] Fix zookeeper dir with mesos conf and add docs.
Fix zookeeper dir configuration used in cluster mode, and also add documentation around these settings.
Author: Timothy Chen <tnachen@gmail.com>
Closes #10057 from tnachen/fix_mesos_dir.
Diffstat (limited to 'docs/running-on-mesos.md')
| -rw-r--r-- | docs/running-on-mesos.md | 5 |
1 files changed, 4 insertions, 1 deletions
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index ed720f1039..0ef1ccb36e 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -153,7 +153,10 @@ can find the results of the driver from the Mesos Web UI. To use cluster mode, you must start the `MesosClusterDispatcher` in your cluster via the `sbin/start-mesos-dispatcher.sh` script, passing in the Mesos master URL (e.g: mesos://host:5050). This starts the `MesosClusterDispatcher` as a daemon running on the host. -If you like to run the `MesosClusterDispatcher` with Marathon, you need to run the `MesosClusterDispatcher` in the foreground (i.e: `bin/spark-class org.apache.spark.deploy.mesos.MesosClusterDispatcher`). +If you like to run the `MesosClusterDispatcher` with Marathon, you need to run the `MesosClusterDispatcher` in the foreground (i.e: `bin/spark-class org.apache.spark.deploy.mesos.MesosClusterDispatcher`). Note that the `MesosClusterDispatcher` not yet supports multiple instances for HA. + +The `MesosClusterDispatcher` also supports writing recovery state into Zookeeper. This will allow the `MesosClusterDispatcher` to be able to recover all submitted and running containers on relaunch. In order to enable this recovery mode, you can set SPARK_DAEMON_JAVA_OPTS in spark-env by configuring `spark.deploy.recoveryMode` and related spark.deploy.zookeeper.* configurations. +For more information about these configurations please refer to the configurations (doc)[configurations.html#deploy]. From the client, you can submit a job to Mesos cluster by running `spark-submit` and specifying the master URL to the URL of the `MesosClusterDispatcher` (e.g: mesos://dispatcher:7077). You can view driver statuses on the |