diff options
| author | mweindel <m.weindel@usu-software.de> | 2015-04-21 20:19:33 -0400 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-04-21 20:19:33 -0400 |
| commit | b063a61b9852cf9b9d2c905332d2ecb2fd716cc4 (patch) | |
| tree | 89a708189f271d326bb52849230468edf1205ffc /core | |
| parent | ae036d08170202074b266afd17ce34b689c70b0c (diff) | |
| download | spark-b063a61b9852cf9b9d2c905332d2ecb2fd716cc4.tar.gz spark-b063a61b9852cf9b9d2c905332d2ecb2fd716cc4.tar.bz2 spark-b063a61b9852cf9b9d2c905332d2ecb2fd716cc4.zip | |
Avoid warning message about invalid refuse_seconds value in Mesos >=0.21...
Starting with version 0.21.0, Apache Mesos is very noisy if the filter parameter refuse_seconds is set to an invalid value like `-1`.
I have seen systems with millions of log lines like
```
W0420 18:00:48.773059 32352 hierarchical_allocator_process.hpp:589] Using the default value of 'refuse_seconds' to create the refused resources filter because the input value is negative
```
in the Mesos master INFO and WARNING log files.
Therefore the CoarseMesosSchedulerBackend should set the default value for refuse seconds (i.e. 5 seconds) directly.
This is no problem for the fine-grained MesosSchedulerBackend, as it uses the value 1 second for this parameter.
Author: mweindel <m.weindel@usu-software.de>
Closes #5597 from MartinWeindel/master and squashes the following commits:
2f99ffd [mweindel] Avoid warning message about invalid refuse_seconds value in Mesos >=0.21.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala index b037a4966c..82f652dae0 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala @@ -207,7 +207,7 @@ private[spark] class CoarseMesosSchedulerBackend( */ override def resourceOffers(d: SchedulerDriver, offers: JList[Offer]) { synchronized { - val filters = Filters.newBuilder().setRefuseSeconds(-1).build() + val filters = Filters.newBuilder().setRefuseSeconds(5).build() for (offer <- offers) { val slaveId = offer.getSlaveId.toString |