diff options
| author | Sandy Ryza <sandy@cloudera.com> | 2014-12-09 11:02:43 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-12-09 11:02:43 -0800 |
| commit | 912563aa3553afc0871d5b5858f533aa39cb99e5 (patch) | |
| tree | 092241ac4c78deef8053f5095bc9680b3c8532cd /docs/building-spark.md | |
| parent | 383c5555c9f26c080bc9e3a463aab21dd5b3797f (diff) | |
| download | spark-912563aa3553afc0871d5b5858f533aa39cb99e5.tar.gz spark-912563aa3553afc0871d5b5858f533aa39cb99e5.tar.bz2 spark-912563aa3553afc0871d5b5858f533aa39cb99e5.zip | |
SPARK-4338. [YARN] Ditch yarn-alpha.
Sorry if this is a little premature with 1.2 still not out the door, but it will make other work like SPARK-4136 and SPARK-2089 a lot easier.
Author: Sandy Ryza <sandy@cloudera.com>
Closes #3215 from sryza/sandy-spark-4338 and squashes the following commits:
1c5ac08 [Sandy Ryza] Update building Spark docs and remove unnecessary newline
9c1421c [Sandy Ryza] SPARK-4338. Ditch yarn-alpha.
Diffstat (limited to 'docs/building-spark.md')
| -rw-r--r-- | docs/building-spark.md | 25 |
1 files changed, 2 insertions, 23 deletions
diff --git a/docs/building-spark.md b/docs/building-spark.md index 6cca2da8e8..4922e877e9 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -60,32 +60,11 @@ mvn -Dhadoop.version=2.0.0-mr1-cdh4.2.0 -DskipTests clean package mvn -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean package {% endhighlight %} -For Apache Hadoop 2.x, 0.23.x, Cloudera CDH, and other Hadoop versions with YARN, you can enable the "yarn-alpha" or "yarn" profile and optionally set the "yarn.version" property if it is different from "hadoop.version". The additional build profile required depends on the YARN version: - -<table class="table"> - <thead> - <tr><th>YARN version</th><th>Profile required</th></tr> - </thead> - <tbody> - <tr><td>0.23.x to 2.1.x</td><td>yarn-alpha (Deprecated.)</td></tr> - <tr><td>2.2.x and later</td><td>yarn</td></tr> - </tbody> -</table> - -Note: Support for YARN-alpha API's will be removed in Spark 1.3 (see SPARK-3445). +For Apache Hadoop 2.x, 0.23.x, Cloudera CDH, and other Hadoop versions with YARN, you can enable the "yarn" profile and optionally set the "yarn.version" property if it is different from "hadoop.version". As of Spark 1.3, Spark only supports YARN versions 2.2.0 and later. Examples: {% highlight bash %} -# Apache Hadoop 2.0.5-alpha -mvn -Pyarn-alpha -Dhadoop.version=2.0.5-alpha -DskipTests clean package - -# Cloudera CDH 4.2.0 -mvn -Pyarn-alpha -Dhadoop.version=2.0.0-cdh4.2.0 -DskipTests clean package - -# Apache Hadoop 0.23.x -mvn -Pyarn-alpha -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean package - # Apache Hadoop 2.2.X mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -DskipTests clean package @@ -99,7 +78,7 @@ Versions of Hadoop after 2.5.X may or may not work with the -Phadoop-2.4 profile released after this version of Spark). # Different versions of HDFS and YARN. -mvn -Pyarn-alpha -Phadoop-2.3 -Dhadoop.version=2.3.0 -Dyarn.version=0.23.7 -DskipTests clean package +mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Dyarn.version=2.2.0 -DskipTests clean package {% endhighlight %} # Building With Hive and JDBC Support |