diff options
author | Holden Karau <holden@pigscanfly.ca> | 2014-01-05 22:05:30 -0800 |
---|---|---|
committer | Holden Karau <holden@pigscanfly.ca> | 2014-01-05 22:05:30 -0800 |
commit | d86dc74d796121b61ff43c632791c52dd49ff8ad (patch) | |
tree | b04601ff15a651093d3a00e54c8f0e4630c72505 /README.md | |
parent | df92f1c0254dc9073c18bc7b76f8b9523ecd7cec (diff) | |
download | spark-d86dc74d796121b61ff43c632791c52dd49ff8ad.tar.gz spark-d86dc74d796121b61ff43c632791c52dd49ff8ad.tar.bz2 spark-d86dc74d796121b61ff43c632791c52dd49ff8ad.zip |
Code review feedback
Diffstat (limited to 'README.md')
-rw-r--r-- | README.md | 14 |
1 files changed, 8 insertions, 6 deletions
@@ -13,7 +13,9 @@ This README file only contains basic setup instructions. ## Building Spark requires Scala 2.10. The project is built using Simple Build Tool (SBT), -which can be obtained [here](http://www.scala-sbt.org). To build Spark and its example programs, run: +which can be obtained [here](http://www.scala-sbt.org). If SBT is installed we +will use the system version of sbt otherwise we will attempt to download it +automatically. To build Spark and its example programs, run: ./sbt/sbt assembly @@ -55,22 +57,22 @@ For Apache Hadoop versions 1.x, Cloudera CDH MRv1, and other Hadoop versions without YARN, use: # Apache Hadoop 1.2.1 - $ SPARK_HADOOP_VERSION=1.2.1 sbt assembly + $ SPARK_HADOOP_VERSION=1.2.1 sbt/sbt assembly # Cloudera CDH 4.2.0 with MapReduce v1 - $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt assembly + $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly For Apache Hadoop 2.2.X, 2.1.X, 2.0.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions with YARN, also set `SPARK_YARN=true`: # Apache Hadoop 2.0.5-alpha - $ SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt assembly + $ SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt/sbt assembly # Cloudera CDH 4.2.0 with MapReduce v2 - $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt assembly + $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly # Apache Hadoop 2.2.X and newer - $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt assembly + $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly When developing a Spark application, specify the Hadoop version by adding the "hadoop-client" artifact to your project's dependencies. For example, if you're |