diff options
| author | jerryshao <saisai.shao@intel.com> | 2015-07-14 19:54:02 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-07-14 19:54:02 -0700 |
| commit | bb870e72f42b6ce8d056df259f6fcf41808d7ed2 (patch) | |
| tree | 1f91b06b141770b4f514ea1782243548ed31f50b /dev/create-release/known_translations | |
| parent | f957796c4b3c3cd95edfc64500a045f7e810ee87 (diff) | |
| download | spark-bb870e72f42b6ce8d056df259f6fcf41808d7ed2.tar.gz spark-bb870e72f42b6ce8d056df259f6fcf41808d7ed2.tar.bz2 spark-bb870e72f42b6ce8d056df259f6fcf41808d7ed2.zip | |
[SPARK-5523] [CORE] [STREAMING] Add a cache for hostname in TaskMetrics to decrease the memory usage and GC overhead
Hostname in TaskMetrics will be created through deserialization, mostly the number of hostname is only the order of number of cluster node, so adding a cache layer to dedup the object could reduce the memory usage and alleviate GC overhead, especially for long-running and fast job generation applications like Spark Streaming.
Author: jerryshao <saisai.shao@intel.com>
Author: Saisai Shao <saisai.shao@intel.com>
Closes #5064 from jerryshao/SPARK-5523 and squashes the following commits:
3e2412a [jerryshao] Address the comments
b092a81 [Saisai Shao] Add a pool to cache the hostname
Diffstat (limited to 'dev/create-release/known_translations')
0 files changed, 0 insertions, 0 deletions