diff options
author | Shivaram Venkataraman <shivaram@eecs.berkeley.edu> | 2014-03-25 13:05:30 -0700 |
---|---|---|
committer | Kay Ousterhout <kayousterhout@gmail.com> | 2014-03-25 13:05:30 -0700 |
commit | f8111eaeb0e35f6aa9b1e3ec1173fff207174155 (patch) | |
tree | 19ddfc891045437de89124055f6ff3344ae70530 /docs | |
parent | 71d4ed271bcbddb154643bd44297ed77190e75cf (diff) | |
download | spark-f8111eaeb0e35f6aa9b1e3ec1173fff207174155.tar.gz spark-f8111eaeb0e35f6aa9b1e3ec1173fff207174155.tar.bz2 spark-f8111eaeb0e35f6aa9b1e3ec1173fff207174155.zip |
SPARK-1319: Fix scheduler to account for tasks using > 1 CPUs.
Move CPUS_PER_TASK to TaskSchedulerImpl as the value is a constant and use it in both Mesos and CoarseGrained scheduler backends.
Thanks @kayousterhout for the design discussion
Author: Shivaram Venkataraman <shivaram@eecs.berkeley.edu>
Closes #219 from shivaram/multi-cpus and squashes the following commits:
5c7d685 [Shivaram Venkataraman] Don't pass availableCpus to TaskSetManager
260e4d5 [Shivaram Venkataraman] Add a check for non-zero CPUs in TaskSetManager
73fcf6f [Shivaram Venkataraman] Add documentation for spark.task.cpus
647bc45 [Shivaram Venkataraman] Fix scheduler to account for tasks using > 1 CPUs. Move CPUS_PER_TASK to TaskSchedulerImpl as the value is a constant and use it in both Mesos and CoarseGrained scheduler backends.
Diffstat (limited to 'docs')
-rw-r--r-- | docs/configuration.md | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 16ee5ec0f2..1ff0150567 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -579,6 +579,13 @@ Apart from these, the following properties are also available, and may be useful out and giving up. </td> </tr> +<tr> + <td>spark.task.cpus</td> + <td>1</td> + <td> + Number of cores to allocate for each task. + </td> +</tr> </table> ## Viewing Spark Properties |