diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2013-12-15 20:30:21 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2013-12-16 21:53:51 -0800 |
| commit | c6f95e603e2c58b2fab6255c824a839807f03026 (patch) | |
| tree | 7b11d9a3374b1b86dbf318832ef11b6bfe875e44 /README.md | |
| parent | 964a3b6971716823f473f672611951d1e489a552 (diff) | |
| download | spark-c6f95e603e2c58b2fab6255c824a839807f03026.tar.gz spark-c6f95e603e2c58b2fab6255c824a839807f03026.tar.bz2 spark-c6f95e603e2c58b2fab6255c824a839807f03026.zip | |
Attempt with extra repositories
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 6 |
1 files changed, 2 insertions, 4 deletions
@@ -54,7 +54,7 @@ versions without YARN, use: # Cloudera CDH 4.2.0 with MapReduce v1 $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly -For Apache Hadoop 2.0.X, 2.1.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions +For Apache Hadoop 2.2.X, 2.1.X, 2.0.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions with YARN, also set `SPARK_YARN=true`: # Apache Hadoop 2.0.5-alpha @@ -63,10 +63,8 @@ with YARN, also set `SPARK_YARN=true`: # Cloudera CDH 4.2.0 with MapReduce v2 $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly -When building for Hadoop 2.2.X and newer, you'll need to include the additional `new-yarn` profile: - # Apache Hadoop 2.2.X and newer - $ mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn + $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly When developing a Spark application, specify the Hadoop version by adding the "hadoop-client" artifact to your project's dependencies. For example, if you're |