diff options
| author | lianhuiwang <lianhuiwang09@gmail.com> | 2015-02-06 14:47:52 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-02-06 14:48:46 -0800 |
| commit | af6ddf8b61c34259984386aa3415157e17bfd49a (patch) | |
| tree | ecddc8102d2d2fb152edc00eae465ad66db7491b /data | |
| parent | caca15a4c0f3779f77fd23bb161b671d402359a8 (diff) | |
| download | spark-af6ddf8b61c34259984386aa3415157e17bfd49a.tar.gz spark-af6ddf8b61c34259984386aa3415157e17bfd49a.tar.bz2 spark-af6ddf8b61c34259984386aa3415157e17bfd49a.zip | |
[SPARK-4994][network]Cleanup removed executors' ShuffleInfo in yarn shuffle service
when the application is completed, yarn's nodemanager can remove application's local-dirs.but all executors' metadata of completed application havenot be removed. now it lets yarn ShuffleService to have much more memory to store Executors' ShuffleInfo. so these metadata need to be removed.
Author: lianhuiwang <lianhuiwang09@gmail.com>
Closes #3828 from lianhuiwang/SPARK-4994 and squashes the following commits:
f3ba1d2 [lianhuiwang] Cleanup removed executors' ShuffleInfo
(cherry picked from commit 61073f832128845a76469fc37376483b784c927b)
Signed-off-by: Andrew Or <andrew@databricks.com>
Diffstat (limited to 'data')
0 files changed, 0 insertions, 0 deletions