diff options
author | Davies Liu <davies@databricks.com> | 2015-02-03 22:30:23 -0800 |
---|---|---|
committer | Kay Ousterhout <kayousterhout@gmail.com> | 2015-02-03 22:30:23 -0800 |
commit | 83de71c45bb9f22049243dd7518b679c4e13c2df (patch) | |
tree | 134797e34ea9543357263540bfb8d843080e0228 /bin/pyspark | |
parent | 242b4f02df7f71ebcfa86a85c9ed39e40750a7fd (diff) | |
download | spark-83de71c45bb9f22049243dd7518b679c4e13c2df.tar.gz spark-83de71c45bb9f22049243dd7518b679c4e13c2df.tar.bz2 spark-83de71c45bb9f22049243dd7518b679c4e13c2df.zip |
[SPARK-4939] revive offers periodically in LocalBackend
The locality timeout assume that the SchedulerBackend can revive offers periodically, but currently LocalBackend did do that, then some job with mixed locality levels in local mode will hang forever.
This PR let LocalBackend revive offers periodically, just like in cluster mode.
Author: Davies Liu <davies@databricks.com>
Closes #4147 from davies/revive and squashes the following commits:
2acdf9d [Davies Liu] Update LocalBackend.scala
3c8ca7c [Davies Liu] Update LocalBackend.scala
d1b60d2 [Davies Liu] address comments from Kay
33ac9bb [Davies Liu] fix build
d0da0d5 [Davies Liu] Merge branch 'master' of github.com:apache/spark into revive
6cf5972 [Davies Liu] fix thread-safety
ed62a31 [Davies Liu] fix scala style
df9008b [Davies Liu] fix typo
bfc1396 [Davies Liu] revive offers periodically in LocalBackend
Diffstat (limited to 'bin/pyspark')
0 files changed, 0 insertions, 0 deletions