aboutsummaryrefslogtreecommitdiff
path: root/docs/spark-simple-tutorial.md
diff options
context:
space:
mode:
authorAndy Konwinski <andyk@berkeley.edu>2012-09-02 23:05:40 -0700
committerAndy Konwinski <andyk@berkeley.edu>2012-09-12 13:03:43 -0700
commit16da942d66ad3d460889ffcb08ee8c82b1ea7936 (patch)
treed49349d1376fb070950473658a75a33cf51631e6 /docs/spark-simple-tutorial.md
parenta29ac5f9cf3b63cdb0bdd864dc0fea3d3d8db095 (diff)
downloadspark-16da942d66ad3d460889ffcb08ee8c82b1ea7936.tar.gz
spark-16da942d66ad3d460889ffcb08ee8c82b1ea7936.tar.bz2
spark-16da942d66ad3d460889ffcb08ee8c82b1ea7936.zip
Adding docs directory containing documentation currently on the wiki
which can be compiled via jekyll, using the command `jekyll`. To compile and run a local webserver to serve the doc as a website, run `jekyll --server`.
Diffstat (limited to 'docs/spark-simple-tutorial.md')
-rw-r--r--docs/spark-simple-tutorial.md41
1 files changed, 41 insertions, 0 deletions
diff --git a/docs/spark-simple-tutorial.md b/docs/spark-simple-tutorial.md
new file mode 100644
index 0000000000..9875de62bd
--- /dev/null
+++ b/docs/spark-simple-tutorial.md
@@ -0,0 +1,41 @@
+---
+layout: global
+title: Tutorial - Running a Simple Spark Application
+---
+
+1. Create directory for spark demo:
+
+ ~$ mkdir SparkTest
+
+2. Copy the sbt files in ~/spark/sbt directory:
+
+ ~/SparkTest$ cp -r ../spark/sbt .
+
+3. Edit the ~/SparkTest/sbt/sbt file to look like this:
+
+ #!/bin/bash
+ java -Xmx800M -XX:MaxPermSize=150m -jar $(dirname $0)/sbt-launch-*.jar "$@"
+
+4. To build a Spark application, you need Spark and its dependencies in a single Java archive (JAR) file. Create this JAR in Spark's main directory with sbt as:
+
+ ~/spark$ sbt/sbt assembly
+
+5. create a source file in ~/SparkTest/src/main/scala directory:
+
+ ~/SparkTest/src/main/scala$ vi Test1.scala
+
+6. Make the contain of the Test1.scala file like this:
+
+ import spark.SparkContext
+ import spark.SparkContext._
+ object Test1 {
+ def main(args: Array[String]) {
+ val sc = new SparkContext("local", "SparkTest")
+ println(sc.parallelize(1 to 10).reduce(_ + _))
+ System.exit(0)
+ }
+ }
+
+7. Run the Test1.scala file:
+
+ ~/SparkTest$ sbt/sbt run