aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorPrashant Sharma <scrapcodes@gmail.com>2014-01-02 14:09:37 +0530
committerPrashant Sharma <scrapcodes@gmail.com>2014-01-02 14:09:37 +0530
commit6be4c1119493dea2af9734ad8b59fcded31f2676 (patch)
tree5005141392dfacd0f4afb8cb9f463668a3900287 /README.md
parent8821c3a5262d6893d2a1fd6ed86afd1213114b4d (diff)
downloadspark-6be4c1119493dea2af9734ad8b59fcded31f2676.tar.gz
spark-6be4c1119493dea2af9734ad8b59fcded31f2676.tar.bz2
spark-6be4c1119493dea2af9734ad8b59fcded31f2676.zip
Removed sbt folder and changed docs accordingly
Diffstat (limited to 'README.md')
-rw-r--r--README.md30
1 files changed, 23 insertions, 7 deletions
diff --git a/README.md b/README.md
index 1550a8b551..22e7ab8245 100644
--- a/README.md
+++ b/README.md
@@ -13,9 +13,9 @@ This README file only contains basic setup instructions.
## Building
Spark requires Scala 2.10. The project is built using Simple Build Tool (SBT),
-which is packaged with it. To build Spark and its example programs, run:
+which can be obtained from [here](http://www.scala-sbt.org/release/docs/Getting-Started/Setup.html). To build Spark and its example programs, run:
- sbt/sbt assembly
+ sbt assembly
Once you've built Spark, the easiest way to start using it is the shell:
@@ -36,6 +36,22 @@ All of the Spark samples take a `<master>` parameter that is the cluster URL
to connect to. This can be a mesos:// or spark:// URL, or "local" to run
locally with one thread, or "local[N]" to run locally with N threads.
+## Running tests
+
+### With sbt. (you need sbt installed)
+Once you have built spark with `sbt assembly` mentioned in [Building](#Building) section. Test suits can be run as follows on *nix based systems using sbt.
+
+`SPARK_HOME=$(pwd) SPARK_TESTING=1 sbt test`
+
+TODO: figure out instructions for windows.
+
+### With maven.
+
+1. Build assembly by
+`mvn package -DskipTests`
+
+2. Run tests
+`mvn test`
## A Note About Hadoop Versions
@@ -49,22 +65,22 @@ For Apache Hadoop versions 1.x, Cloudera CDH MRv1, and other Hadoop
versions without YARN, use:
# Apache Hadoop 1.2.1
- $ SPARK_HADOOP_VERSION=1.2.1 sbt/sbt assembly
+ $ SPARK_HADOOP_VERSION=1.2.1 sbt assembly
# Cloudera CDH 4.2.0 with MapReduce v1
- $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly
+ $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt assembly
For Apache Hadoop 2.2.X, 2.1.X, 2.0.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions
with YARN, also set `SPARK_YARN=true`:
# Apache Hadoop 2.0.5-alpha
- $ SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt/sbt assembly
+ $ SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt assembly
# Cloudera CDH 4.2.0 with MapReduce v2
- $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly
+ $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt assembly
# Apache Hadoop 2.2.X and newer
- $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly
+ $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt assembly
When developing a Spark application, specify the Hadoop version by adding the
"hadoop-client" artifact to your project's dependencies. For example, if you're