diff options
-rw-r--r-- | .gitignore | 2 | ||||
-rw-r--r-- | README | 6 | ||||
-rw-r--r-- | conf/java-opts | 0 | ||||
-rwxr-xr-x | conf/spark-env.sh | 13 |
4 files changed, 20 insertions, 1 deletions
diff --git a/.gitignore b/.gitignore index 98dfe5a27b..8156d6e8c1 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,5 @@ work .DS_Store third_party/libmesos.so third_party/libmesos.dylib +conf/java-opts +conf/spark-env.sh @@ -5,7 +5,7 @@ Spark requires Scala 2.8. This version has been tested with 2.8.0.final. To build and run Spark, you will need to have Scala's bin in your $PATH, or you will need to set the SCALA_HOME environment variable to point to where you've installed Scala. Scala must be accessible through one -of these methods on Nexus slave nodes as well as on the master. +of these methods on Mesos slave nodes as well as on the master. To build Spark and the example programs, run make. @@ -13,6 +13,10 @@ To run one of the examples, use ./run <class> <params>. For example, ./run SparkLR will run the Logistic Regression example. Each of the example programs prints usage help if no params are given. +All of the Spark samples take a <host> parameter that is the Mesos master +to connect to. This can be a Mesos URL, or "local" to run locally with one +thread, or "local[N]" to run locally with N threads. + Tip: If you are building Spark and examples repeatedly, export USE_FSC=1 to have the Makefile use the fsc compiler daemon instead of scalac. diff --git a/conf/java-opts b/conf/java-opts new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/conf/java-opts diff --git a/conf/spark-env.sh b/conf/spark-env.sh new file mode 100755 index 0000000000..6852b23a34 --- /dev/null +++ b/conf/spark-env.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash + +# Set Spark environment variables for your site in this file. Some useful +# variables to set are: +# - MESOS_HOME, to point to your Mesos installation +# - SCALA_HOME, to point to your Scala installation +# - SPARK_CLASSPATH, to add elements to Spark's classpath +# - SPARK_JAVA_OPTS, to add JVM options +# - SPARK_MEM, to change the amount of memory used per node (this should +# be in the same format as the JVM's -Xmx option, e.g. 300m or 1g). +# - SPARK_LIBRARY_PATH, to add extra search paths for native libraries. + + |