diff options
| author | Aaron Davidson <aaron@databricks.com> | 2014-11-26 00:32:45 -0500 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-11-26 00:32:55 -0500 |
| commit | c7185f0c08e2a42e2595466e2d8ac394cbf66f5b (patch) | |
| tree | 2698b56fe8226ada4654ec0ef9909d01fa08ac13 /external/flume | |
| parent | 380eba5f49eca1dbd4084e6c84e19866fffd4efa (diff) | |
| download | spark-c7185f0c08e2a42e2595466e2d8ac394cbf66f5b.tar.gz spark-c7185f0c08e2a42e2595466e2d8ac394cbf66f5b.tar.bz2 spark-c7185f0c08e2a42e2595466e2d8ac394cbf66f5b.zip | |
[SPARK-4516] Avoid allocating Netty PooledByteBufAllocators unnecessarily
Turns out we are allocating an allocator pool for every TransportClient (which means that the number increases with the number of nodes in the cluster), when really we should just reuse one for all clients.
This patch, as expected, greatly decreases off-heap memory allocation, and appears to make allocation only proportional to the number of cores.
Author: Aaron Davidson <aaron@databricks.com>
Closes #3465 from aarondav/fewer-pools and squashes the following commits:
36c49da [Aaron Davidson] [SPARK-4516] Avoid allocating unnecessarily Netty PooledByteBufAllocators
(cherry picked from commit 346bc17a2ec8fc9e6eaff90733aa1e8b6b46883e)
Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Diffstat (limited to 'external/flume')
0 files changed, 0 insertions, 0 deletions