diff options
| author | xiaojian.fxj <xiaojian.fxj@alibaba-inc.com> | 2017-03-12 10:29:00 -0700 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2017-03-12 10:29:00 -0700 |
| commit | 2f5187bde1544c452fe5116a2bd243653332a079 (patch) | |
| tree | 5c3d32d0b6e49cbca18499e5444d3f1015293dad /R/pkg | |
| parent | e29a74d5b1fa3f9356b7af5dd7e3fce49bc8eb7d (diff) | |
| download | spark-2f5187bde1544c452fe5116a2bd243653332a079.tar.gz spark-2f5187bde1544c452fe5116a2bd243653332a079.tar.bz2 spark-2f5187bde1544c452fe5116a2bd243653332a079.zip | |
[SPARK-19831][CORE] Reuse the existing cleanupThreadExecutor to clean up the directories of finished applications to avoid the block
Cleaning the application may cost much time at worker, then it will block that the worker send heartbeats master because the worker is extend ThreadSafeRpcEndpoint. If the heartbeat from a worker is blocked by the message ApplicationFinished, master will think the worker is dead. If the worker has a driver, the driver will be scheduled by master again.
It had better reuse the existing cleanupThreadExecutor to clean up the directories of finished applications to avoid the block.
Author: xiaojian.fxj <xiaojian.fxj@alibaba-inc.com>
Closes #17189 from hustfxj/worker-hearbeat.
Diffstat (limited to 'R/pkg')
0 files changed, 0 insertions, 0 deletions