diff options
| author | Adam Budde <budde@amazon.com> | 2016-02-02 19:35:33 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-02-02 19:35:33 -0800 |
| commit | ff71261b651a7b289ea2312abd6075da8b838ed9 (patch) | |
| tree | 8807de2ed8a8dc00b345d852bff46e880b02aef2 /licenses/LICENSE-boto.txt | |
| parent | 21112e8a14c042ccef4312079672108a1082a95e (diff) | |
| download | spark-ff71261b651a7b289ea2312abd6075da8b838ed9.tar.gz spark-ff71261b651a7b289ea2312abd6075da8b838ed9.tar.bz2 spark-ff71261b651a7b289ea2312abd6075da8b838ed9.zip | |
[SPARK-13122] Fix race condition in MemoryStore.unrollSafely()
https://issues.apache.org/jira/browse/SPARK-13122
A race condition can occur in MemoryStore's unrollSafely() method if two threads that
return the same value for currentTaskAttemptId() execute this method concurrently. This
change makes the operation of reading the initial amount of unroll memory used, performing
the unroll, and updating the associated memory maps atomic in order to avoid this race
condition.
Initial proposed fix wraps all of unrollSafely() in a memoryManager.synchronized { } block. A cleaner approach might be introduce a mechanism that synchronizes based on task attempt ID. An alternative option might be to track unroll/pending unroll memory based on block ID rather than task attempt ID.
Author: Adam Budde <budde@amazon.com>
Closes #11012 from budde/master.
Diffstat (limited to 'licenses/LICENSE-boto.txt')
0 files changed, 0 insertions, 0 deletions