diff options
| author | Prashant Sharma <prashant.s@imaginea.com> | 2013-12-11 10:21:53 +0530 |
|---|---|---|
| committer | Prashant Sharma <prashant.s@imaginea.com> | 2013-12-11 10:21:53 +0530 |
| commit | 603af51bb5257744ce0db28e7f10db6a2ba899ec (patch) | |
| tree | 31ad611d8d56147ec90108d2b3a92d38eae4d150 /README.md | |
| parent | 17db6a9041d5e83d7b6fe47f9c36758d0613fcd6 (diff) | |
| parent | d2efe13574090e93c600adeacc7f6356bc196e6c (diff) | |
| download | spark-603af51bb5257744ce0db28e7f10db6a2ba899ec.tar.gz spark-603af51bb5257744ce0db28e7f10db6a2ba899ec.tar.bz2 spark-603af51bb5257744ce0db28e7f10db6a2ba899ec.zip | |
Merge branch 'master' into akka-bug-fix
Conflicts:
core/pom.xml
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
pom.xml
project/SparkBuild.scala
streaming/pom.xml
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 8 |
1 files changed, 5 insertions, 3 deletions
@@ -54,7 +54,7 @@ versions without YARN, use: # Cloudera CDH 4.2.0 with MapReduce v1 $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly -For Apache Hadoop 2.x, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions +For Apache Hadoop 2.0.X, 2.1.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions with YARN, also set `SPARK_YARN=true`: # Apache Hadoop 2.0.5-alpha @@ -63,8 +63,10 @@ with YARN, also set `SPARK_YARN=true`: # Cloudera CDH 4.2.0 with MapReduce v2 $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly -For convenience, these variables may also be set through the `conf/spark-env.sh` file -described below. +When building for Hadoop 2.2.X and newer, you'll need to include the additional `new-yarn` profile: + + # Apache Hadoop 2.2.X and newer + $ mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn When developing a Spark application, specify the Hadoop version by adding the "hadoop-client" artifact to your project's dependencies. For example, if you're |