diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-01-08 00:38:31 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-01-08 00:38:31 -0800 |
| commit | bc81ce040ded3fe791ecffb15baa408346afc92a (patch) | |
| tree | 3330fddcffbca7bd8f42f2fa96ded31cdb0430f9 /docs/spark-standalone.md | |
| parent | 3ec21f2eee0a322bcb5cce9678a816b012227a61 (diff) | |
| parent | bb6a39a6872fb8e7118c5a402b7b9d67459e9a64 (diff) | |
| download | spark-bc81ce040ded3fe791ecffb15baa408346afc92a.tar.gz spark-bc81ce040ded3fe791ecffb15baa408346afc92a.tar.bz2 spark-bc81ce040ded3fe791ecffb15baa408346afc92a.zip | |
Merge remote-tracking branch 'apache-github/master' into standalone-driver
Conflicts:
core/src/test/scala/org/apache/spark/deploy/JsonProtocolSuite.scala
pom.xml
Diffstat (limited to 'docs/spark-standalone.md')
| -rw-r--r-- | docs/spark-standalone.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/spark-standalone.md b/docs/spark-standalone.md index ecd642cc60..2a186261b7 100644 --- a/docs/spark-standalone.md +++ b/docs/spark-standalone.md @@ -195,6 +195,16 @@ val conf = new SparkConf() val sc = new SparkContext(conf) {% endhighlight %} +In addition, you can configure `spark.deploy.defaultCores` on the cluster master process to change the +default for applications that don't set `spark.cores.max` to something less than infinite. +Do this by adding the following to `conf/spark-env.sh`: + +{% highlight bash %} +export SPARK_JAVA_OPTS="-Dspark.deploy.defaultCores=<value>" +{% endhighlight %} + +This is useful on shared clusters where users might not have configured a maximum number of cores +individually. # Monitoring and Logging |