diff options
| author | Brenden Matthews <brenden@diddyinc.com> | 2014-11-05 16:02:44 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-11-05 16:03:06 -0800 |
| commit | c58c1bb830c9efdecd680d19228dca56362b90b5 (patch) | |
| tree | 636f6d9a43cf988cbac875bf86e8b1ce1e04ae18 /core/src/main | |
| parent | 590a94373d237f727f58cfd6fd02135b44d964f8 (diff) | |
| download | spark-c58c1bb830c9efdecd680d19228dca56362b90b5.tar.gz spark-c58c1bb830c9efdecd680d19228dca56362b90b5.tar.bz2 spark-c58c1bb830c9efdecd680d19228dca56362b90b5.zip | |
[SPARK-4158] Fix for missing resources.
Mesos offers may not contain all resources, and Spark needs to check to
ensure they are present and sufficient. Spark may throw an erroneous
exception when resources aren't present.
Author: Brenden Matthews <brenden@diddyinc.com>
Closes #3024 from brndnmtthws/fix-mesos-resource-misuse and squashes the following commits:
e5f9580 [Brenden Matthews] [SPARK-4158] Fix for missing resources.
(cherry picked from commit cb0eae3b78d7f6f56c0b9521ee48564a4967d3de)
Signed-off-by: Andrew Or <andrew@databricks.com>
Diffstat (limited to 'core/src/main')
2 files changed, 2 insertions, 4 deletions
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala index 43c0d8cb7e..9cdde20883 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala @@ -235,8 +235,7 @@ private[spark] class CoarseMesosSchedulerBackend( for (r <- res if r.getName == name) { return r.getScalar.getValue } - // If we reached here, no resource with the required name was present - throw new IllegalArgumentException("No resource called " + name + " in " + res) + 0 } /** Build a Mesos resource protobuf object */ diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala index ea6b06c7f6..a98d09de87 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala @@ -273,8 +273,7 @@ private[spark] class MesosSchedulerBackend( for (r <- res if r.getName == name) { return r.getScalar.getValue } - // If we reached here, no resource with the required name was present - throw new IllegalArgumentException("No resource called " + name + " in " + res) + 0 } /** Turn a Spark TaskDescription into a Mesos task */ |