diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2016-11-30 14:47:41 -0500 |
|---|---|---|
| committer | Andrew Or <andrewor14@gmail.com> | 2016-11-30 14:47:41 -0500 |
| commit | c51c7725944d60738e2bac3e11f6aea74812905c (patch) | |
| tree | a40be2277223c975e1b8875d0ff521f66d3a7f54 /core/src/test/scala | |
| parent | bc95ea0be5b880673d452f5eec47fbfd403d94ce (diff) | |
| download | spark-c51c7725944d60738e2bac3e11f6aea74812905c.tar.gz spark-c51c7725944d60738e2bac3e11f6aea74812905c.tar.bz2 spark-c51c7725944d60738e2bac3e11f6aea74812905c.zip | |
[SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors
## What changes were proposed in this pull request?
The method `TaskSchedulerImpl.runningTasksByExecutors()` accesses the mutable `executorIdToRunningTaskIds` map without proper synchronization. In addition, as markhamstra pointed out in #15986, the signature's use of parentheses is a little odd given that this is a pure getter method.
This patch fixes both issues.
## How was this patch tested?
Covered by existing tests.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #16073 from JoshRosen/runningTasksByExecutors-thread-safety.
Diffstat (limited to 'core/src/test/scala')
| -rw-r--r-- | core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala b/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala index 59bea27596..a0b6268331 100644 --- a/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala +++ b/core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala @@ -678,7 +678,7 @@ class TaskSchedulerImplSuite extends SparkFunSuite with LocalSparkContext with B // Check that state associated with the lost task attempt is cleaned up: assert(taskScheduler.taskIdToExecutorId.isEmpty) assert(taskScheduler.taskIdToTaskSetManager.isEmpty) - assert(taskScheduler.runningTasksByExecutors().get("executor0").isEmpty) + assert(taskScheduler.runningTasksByExecutors.get("executor0").isEmpty) } test("if a task finishes with TaskState.LOST its executor is marked as dead") { @@ -709,7 +709,7 @@ class TaskSchedulerImplSuite extends SparkFunSuite with LocalSparkContext with B // Check that state associated with the lost task attempt is cleaned up: assert(taskScheduler.taskIdToExecutorId.isEmpty) assert(taskScheduler.taskIdToTaskSetManager.isEmpty) - assert(taskScheduler.runningTasksByExecutors().get("executor0").isEmpty) + assert(taskScheduler.runningTasksByExecutors.get("executor0").isEmpty) // Check that the executor has been marked as dead assert(!taskScheduler.isExecutorAlive("executor0")) |