diff options
| author | Lianhui Wang <lianhuiwang09@gmail.com> | 2015-11-10 10:40:08 -0800 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2015-11-10 10:40:08 -0800 |
| commit | 6e5fc37883ed81c3ee2338145a48de3036d19399 (patch) | |
| tree | 67c33f795a8764706e7598f4afcae3eb00f42c41 /tags | |
| parent | 689386b1c60997e4505749915f7005a52c207de2 (diff) | |
| download | spark-6e5fc37883ed81c3ee2338145a48de3036d19399.tar.gz spark-6e5fc37883ed81c3ee2338145a48de3036d19399.tar.bz2 spark-6e5fc37883ed81c3ee2338145a48de3036d19399.zip | |
[SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle
with yarn's external shuffle, ExternalShuffleClient of executors reserve its connections for yarn's NodeManager until application has been completed. so it will make NodeManager and executors have many socket connections.
in order to reduce network pressure of NodeManager's shuffleService, after registerWithShuffleServer or fetchBlocks have been completed in ExternalShuffleClient, connection for NM's shuffleService needs to be closed.andrewor14 rxin vanzin
Author: Lianhui Wang <lianhuiwang09@gmail.com>
Closes #9227 from lianhuiwang/spark-11252.
Diffstat (limited to 'tags')
0 files changed, 0 insertions, 0 deletions