diff options
| author | Andrew Or <andrewor14@gmail.com> | 2014-05-14 00:54:33 -0700 |
|---|---|---|
| committer | Aaron Davidson <aaron@databricks.com> | 2014-05-14 00:54:33 -0700 |
| commit | 69f750228f3ec8537a93da08e712596fa8004143 (patch) | |
| tree | 6e2f59a35acc3b2cc140d86f38210c21a4fa1224 /examples/src | |
| parent | 54ae8328bd7d052ba347768cfb02cb5dfdd8045e (diff) | |
| download | spark-69f750228f3ec8537a93da08e712596fa8004143.tar.gz spark-69f750228f3ec8537a93da08e712596fa8004143.tar.bz2 spark-69f750228f3ec8537a93da08e712596fa8004143.zip | |
[SPARK-1769] Executor loss causes NPE race condition
This PR replaces the Schedulable data structures in Pool.scala with thread-safe ones from java. Note that Scala's `with SynchronizedBuffer` trait is soon to be deprecated in 2.11 because it is ["inherently unreliable"](http://www.scala-lang.org/api/2.11.0/index.html#scala.collection.mutable.SynchronizedBuffer). We should slowly drift away from `SynchronizedBuffer` in other places too.
Note that this PR introduces an API-breaking change; `sc.getAllPools` now returns an Array rather than an ArrayBuffer. This is because we want this method to return an immutable copy rather than one may potentially confuse the user if they try to modify the copy, which takes no effect on the original data structure.
Author: Andrew Or <andrewor14@gmail.com>
Closes #762 from andrewor14/pool-npe and squashes the following commits:
383e739 [Andrew Or] JavaConverters -> JavaConversions
3f32981 [Andrew Or] Merge branch 'master' of github.com:apache/spark into pool-npe
769be19 [Andrew Or] Assorted minor changes
2189247 [Andrew Or] Merge branch 'master' of github.com:apache/spark into pool-npe
05ad9e9 [Andrew Or] Fix test - contains is not the same as containsKey
0921ea0 [Andrew Or] var -> val
07d720c [Andrew Or] Synchronize Schedulable data structures
Diffstat (limited to 'examples/src')
0 files changed, 0 insertions, 0 deletions