diff options
| author | Brenden Matthews <brenden@diddyinc.com> | 2014-11-05 16:02:44 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-11-05 16:02:53 -0800 |
| commit | 9ac5c517b64606db7d6b8ac3b823c3d5a45e0ed0 (patch) | |
| tree | 9d92e37743cb17f930e73edfa0a1f8dab77b13b9 /core | |
| parent | 0e16d3a3dde7a0988dfd8eff05922a1ac917fe28 (diff) | |
| download | spark-9ac5c517b64606db7d6b8ac3b823c3d5a45e0ed0.tar.gz spark-9ac5c517b64606db7d6b8ac3b823c3d5a45e0ed0.tar.bz2 spark-9ac5c517b64606db7d6b8ac3b823c3d5a45e0ed0.zip | |
[SPARK-4158] Fix for missing resources.
Mesos offers may not contain all resources, and Spark needs to check to
ensure they are present and sufficient. Spark may throw an erroneous
exception when resources aren't present.
Author: Brenden Matthews <brenden@diddyinc.com>
Closes #3024 from brndnmtthws/fix-mesos-resource-misuse and squashes the following commits:
e5f9580 [Brenden Matthews] [SPARK-4158] Fix for missing resources.
(cherry picked from commit cb0eae3b78d7f6f56c0b9521ee48564a4967d3de)
Signed-off-by: Andrew Or <andrew@databricks.com>
Diffstat (limited to 'core')
2 files changed, 2 insertions, 4 deletions
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala index e4b8598460..5289661eb8 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala @@ -242,8 +242,7 @@ private[spark] class CoarseMesosSchedulerBackend( for (r <- res if r.getName == name) { return r.getScalar.getValue } - // If we reached here, no resource with the required name was present - throw new IllegalArgumentException("No resource called " + name + " in " + res) + 0 } /** Build a Mesos resource protobuf object */ diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala index 7d097a3a7a..c5f3493477 100644 --- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala +++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala @@ -278,8 +278,7 @@ private[spark] class MesosSchedulerBackend( for (r <- res if r.getName == name) { return r.getScalar.getValue } - // If we reached here, no resource with the required name was present - throw new IllegalArgumentException("No resource called " + name + " in " + res) + 0 } /** Turn a Spark TaskDescription into a Mesos task */ |