diff options
| author | Peter Ableda <abledapeter@gmail.com> | 2016-06-23 09:00:31 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-06-23 09:00:31 -0500 |
| commit | 5bf2889bfcfd776e7ee1369443a0474421a800bd (patch) | |
| tree | 9c0461e1d04a97b64e30c40cff0f595edf8fbd80 /yarn | |
| parent | 5eef1e6c6a8b6202fc6db4a90c4caab5169e86c6 (diff) | |
| download | spark-5bf2889bfcfd776e7ee1369443a0474421a800bd.tar.gz spark-5bf2889bfcfd776e7ee1369443a0474421a800bd.tar.bz2 spark-5bf2889bfcfd776e7ee1369443a0474421a800bd.zip | |
[SPARK-16138] Try to cancel executor requests only if we have at least 1
## What changes were proposed in this pull request?
Adding additional check to if statement
## How was this patch tested?
I built and deployed to internal cluster to observe behaviour. After the change the invalid logging is gone:
```
16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 1 executor(s).
16/06/22 08:46:36 INFO yarn.YarnAllocator: Canceling requests for 1 executor container(s) to have a new desired total 1 executors.
16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
16/06/22 08:47:36 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1.
```
Author: Peter Ableda <abledapeter@gmail.com>
Closes #13850 from peterableda/patch-2.
Diffstat (limited to 'yarn')
| -rw-r--r-- | yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala index 1b80071649..dbdac3369b 100644 --- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala +++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala @@ -352,7 +352,7 @@ private[yarn] class YarnAllocator( logInfo(s"Submitted container request (host: ${hostStr(request)}, capability: $resource)") } - } else if (missing < 0) { + } else if (numPendingAllocate > 0 && missing < 0) { val numToCancel = math.min(numPendingAllocate, -missing) logInfo(s"Canceling requests for $numToCancel executor container(s) to have a new desired " + s"total $targetNumExecutors executors.") |