diff options
| author | Kay Ousterhout <kayousterhout@gmail.com> | 2013-12-20 14:39:03 -0800 |
|---|---|---|
| committer | Kay Ousterhout <kayousterhout@gmail.com> | 2013-12-20 14:39:30 -0800 |
| commit | c06945cfe0717006ae0d44da797a4fbb1a48954d (patch) | |
| tree | 9eddb7d47b5829bdb2f117b5676ecc18b11328c7 /README.md | |
| parent | 9228ec847e841a17c7dff7e75bc2e06bea799ea4 (diff) | |
| parent | 0bc57c576792ba800eca0ec196c92a4d29cb3953 (diff) | |
| download | spark-c06945cfe0717006ae0d44da797a4fbb1a48954d.tar.gz spark-c06945cfe0717006ae0d44da797a4fbb1a48954d.tar.bz2 spark-c06945cfe0717006ae0d44da797a4fbb1a48954d.zip | |
Merge remote branch 'upstream/master' into consolidate_schedulers
Conflicts:
core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 14 |
1 files changed, 4 insertions, 10 deletions
@@ -12,9 +12,8 @@ This README file only contains basic setup instructions. ## Building -Spark requires Scala 2.9.3 (Scala 2.10 is not yet supported). The project is -built using Simple Build Tool (SBT), which is packaged with it. To build -Spark and its example programs, run: +Spark requires Scala 2.10. The project is built using Simple Build Tool (SBT), +which is packaged with it. To build Spark and its example programs, run: sbt/sbt assembly @@ -55,7 +54,7 @@ versions without YARN, use: # Cloudera CDH 4.2.0 with MapReduce v1 $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly -For Apache Hadoop 2.0.X, 2.1.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions +For Apache Hadoop 2.2.X, 2.1.X, 2.0.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions with YARN, also set `SPARK_YARN=true`: # Apache Hadoop 2.0.5-alpha @@ -64,13 +63,8 @@ with YARN, also set `SPARK_YARN=true`: # Cloudera CDH 4.2.0 with MapReduce v2 $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly -When building for Hadoop 2.2.X and newer, you'll need to include the additional `new-yarn` profile: - # Apache Hadoop 2.2.X and newer - $ mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn - -For convenience, these variables may also be set through the `conf/spark-env.sh` file -described below. + $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly When developing a Spark application, specify the Hadoop version by adding the "hadoop-client" artifact to your project's dependencies. For example, if you're |