diff options
| author | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-26 22:54:39 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-26 22:54:39 -0700 |
| commit | ea05fc130b64ce356ab7524a3d5bd1e022cf51b5 (patch) | |
| tree | 551ac8546cb21aa750a0967ef115e16639b0ef64 /docs/running-on-yarn.md | |
| parent | 1ef4f0fbd27e54803f14fed1df541fb341daced8 (diff) | |
| download | spark-ea05fc130b64ce356ab7524a3d5bd1e022cf51b5.tar.gz spark-ea05fc130b64ce356ab7524a3d5bd1e022cf51b5.tar.bz2 spark-ea05fc130b64ce356ab7524a3d5bd1e022cf51b5.zip | |
Updates to standalone cluster, web UI and deploy docs.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 16 |
1 files changed, 12 insertions, 4 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 19e7aede27..081b67ae1e 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -3,14 +3,22 @@ layout: global title: Launching Spark on YARN --- -Spark allows you to launch jobs on an existing [YARN](http://hadoop.apache.org/docs/r2.0.1-alpha/hadoop-yarn/hadoop-yarn-site/YARN.html) cluster. +Spark 0.6 adds experimental support for running over a [YARN (Hadoop +NextGen)](http://hadoop.apache.org/docs/r2.0.1-alpha/hadoop-yarn/hadoop-yarn-site/YARN.html) cluster. +Because YARN depends on version 2.0 of the Hadoop libraries, this currently requires checking out a +separate branch of Spark, called `yarn`, which you can do as follows: + + git clone git://github.com/mesos/spark + cd spark + git checkout -b yarn --track origin/yarn + # Preparations -- In order to distribute Spark within the cluster it must be packaged into a single JAR file. This can be done by running `sbt/sbt assembly` -- Your application code must be packaged into a separate jar file. +- In order to distribute Spark within the cluster, it must be packaged into a single JAR file. This can be done by running `sbt/sbt assembly` +- Your application code must be packaged into a separate JAR file. -If you want to test out the YARN deployment mode, you can use the current spark examples. A `spark-examples_2.9.1-0.6.0-SNAPSHOT.jar` file can be generated by running `sbt/sbt package`. +If you want to test out the YARN deployment mode, you can use the current Spark examples. A `spark-examples_2.9.1-0.6.0-SNAPSHOT.jar` file can be generated by running `sbt/sbt package`. # Launching Spark on YARN |