aboutsummaryrefslogtreecommitdiff
path: root/docs/job-scheduling.md
Commit message (Collapse)AuthorAgeFilesLines
* SPARK-4585. Spark dynamic executor allocation should use minExecutors as...Sandy Ryza2015-02-021-5/+4
| | | | | | | | | | | ... initial number Author: Sandy Ryza <sandy@cloudera.com> Closes #4051 from sryza/sandy-spark-4585 and squashes the following commits: d1dd039 [Sandy Ryza] Add spark.dynamicAllocation.initialNumExecutors and make min and max not required b7c59dc [Sandy Ryza] SPARK-4585. Spark dynamic executor allocation should use minExecutors as initial number
* [SPARK-4915][YARN] Fix classname to be specified for external shuffle service.Tsuyoshi Ozawa2014-12-221-1/+1
| | | | | | | | Author: Tsuyoshi Ozawa <ozawa.tsuyoshi@lab.ntt.co.jp> Closes #3757 from oza/SPARK-4915 and squashes the following commits: 3b0d6d6 [Tsuyoshi Ozawa] Fix classname to be specified for external shuffle service.
* [SPARK-4140] Document dynamic allocationAndrew Or2014-12-191-0/+108
| | | | | | | | | | | | | | | | | | | | Once the external shuffle service is also documented, the dynamic allocation section will link to it. Let me know if the whole dynamic allocation should be moved to its separate page; I personally think the organization might be cleaner that way. This patch builds on top of oza's work in #3689. aarondav pwendell Author: Andrew Or <andrew@databricks.com> Author: Tsuyoshi Ozawa <ozawa.tsuyoshi@gmail.com> Closes #3731 from andrewor14/document-dynamic-allocation and squashes the following commits: 1281447 [Andrew Or] Address a few comments b9843f2 [Andrew Or] Document the configs as well 246fb44 [Andrew Or] Merge branch 'SPARK-4839' of github.com:oza/spark into document-dynamic-allocation 8c64004 [Andrew Or] Add documentation for dynamic allocation (without configs) 6827b56 [Tsuyoshi Ozawa] Fixing a documentation of spark.dynamicAllocation.enabled. 53cff58 [Tsuyoshi Ozawa] Adding a documentation about dynamic resource allocation.
* SPARK-1183. Don't use "worker" to mean executorSandy Ryza2014-03-131-2/+2
| | | | | | | | | | | | Author: Sandy Ryza <sandy@cloudera.com> Closes #120 from sryza/sandy-spark-1183 and squashes the following commits: 5066a4a [Sandy Ryza] Remove "worker" in a couple comments 0bd1e46 [Sandy Ryza] Remove --am-class from usage bfc8fe0 [Sandy Ryza] Remove am-class from doc and fix yarn-alpha 607539f [Sandy Ryza] Address review comments 74d087a [Sandy Ryza] SPARK-1183. Don't use "worker" to mean executor
* Add way to limit default # of cores used by applications on standalone modeMatei Zaharia2014-01-071-3/+2
| | | | Also documents the spark.deploy.spreadOut option.
* Updated docs for SparkConf and handled review commentsMatei Zaharia2013-12-301-9/+12
|
* Various broken links in documentationPatrick Wendell2013-12-071-1/+1
|
* Review commentsMatei Zaharia2013-09-081-1/+1
|
* More fair scheduler docs and property names.Matei Zaharia2013-09-081-7/+94
| | | | | Also changed uses of "job" terminology to "application" when they referred to an entire Spark program, to avoid confusion.
* Work in progress:Matei Zaharia2013-09-081-0/+81
- Add job scheduling docs - Rename some fair scheduler properties - Organize intro page better - Link to Apache wiki for "contributing to Spark"