diff options
| -rw-r--r-- | core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala | 4 | ||||
| -rw-r--r-- | docs/configuration.md | 5 |
2 files changed, 7 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala index 65df887477..65cb5016cf 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala @@ -222,6 +222,10 @@ private[spark] class CoarseMesosSchedulerBackend( markRegistered() } + override def sufficientResourcesRegistered(): Boolean = { + totalCoresAcquired >= maxCores * minRegisteredRatio + } + override def disconnected(d: SchedulerDriver) {} override def reregistered(d: SchedulerDriver, masterInfo: MasterInfo) {} diff --git a/docs/configuration.md b/docs/configuration.md index 0b1a273916..1a701f1888 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1112,10 +1112,11 @@ Apart from these, the following properties are also available, and may be useful </tr> <tr> <td><code>spark.scheduler.minRegisteredResourcesRatio</code></td> - <td>0.8 for YARN mode; 0.0 otherwise</td> + <td>0.8 for YARN mode; 0.0 for standalone mode and Mesos coarse-grained mode</td> <td> The minimum ratio of registered resources (registered resources / total expected resources) - (resources are executors in yarn mode, CPU cores in standalone mode) + (resources are executors in yarn mode, CPU cores in standalone mode and Mesos coarsed-grained + mode ['spark.cores.max' value is total expected resources for Mesos coarse-grained mode] ) to wait for before scheduling begins. Specified as a double between 0.0 and 1.0. Regardless of whether the minimum ratio of resources has been reached, the maximum amount of time it will wait before scheduling begins is controlled by config |