diff options
| author | Aaron Davidson <aaron@databricks.com> | 2014-07-03 22:31:41 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-07-03 22:31:41 -0700 |
| commit | 586feb5c9528042420f678f78bacb6c254a5eaf8 (patch) | |
| tree | 50d4af380ce188fd7cd8fa9ffd344afa5eb1fb31 /core/src/main/scala | |
| parent | 5fa0a05763ab1d527efe20e3b10539ac5ffc36de (diff) | |
| download | spark-586feb5c9528042420f678f78bacb6c254a5eaf8.tar.gz spark-586feb5c9528042420f678f78bacb6c254a5eaf8.tar.bz2 spark-586feb5c9528042420f678f78bacb6c254a5eaf8.zip | |
[SPARK-2350] Don't NPE while launching drivers
Prior to this change, we could throw a NPE if we launch a driver while another one is waiting, because removing from an iterator while iterating over it is not safe.
Author: Aaron Davidson <aaron@databricks.com>
Closes #1289 from aarondav/master-fail and squashes the following commits:
1cf1cf4 [Aaron Davidson] SPARK-2350: Don't NPE while launching drivers
Diffstat (limited to 'core/src/main/scala')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/deploy/master/Master.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/deploy/master/Master.scala b/core/src/main/scala/org/apache/spark/deploy/master/Master.scala index 11545b8203..a304102a49 100644 --- a/core/src/main/scala/org/apache/spark/deploy/master/Master.scala +++ b/core/src/main/scala/org/apache/spark/deploy/master/Master.scala @@ -481,7 +481,7 @@ private[spark] class Master( // First schedule drivers, they take strict precedence over applications val shuffledWorkers = Random.shuffle(workers) // Randomization helps balance drivers for (worker <- shuffledWorkers if worker.state == WorkerState.ALIVE) { - for (driver <- waitingDrivers) { + for (driver <- List(waitingDrivers: _*)) { // iterate over a copy of waitingDrivers if (worker.memoryFree >= driver.desc.mem && worker.coresFree >= driver.desc.cores) { launchDriver(worker, driver) waitingDrivers -= driver |