diff options
| author | navis.ryu <navis@apache.org> | 2015-07-21 11:52:52 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-07-21 11:52:52 -0700 |
| commit | 9ba7c64decfc92853bd281e9e7bfb95211080dd4 (patch) | |
| tree | 5ef88207e0923c96752e2ae0b8452ede382b7de6 /project | |
| parent | 87d890cc105a7f41478433b28f53c9aa431db211 (diff) | |
| download | spark-9ba7c64decfc92853bd281e9e7bfb95211080dd4.tar.gz spark-9ba7c64decfc92853bd281e9e7bfb95211080dd4.tar.bz2 spark-9ba7c64decfc92853bd281e9e7bfb95211080dd4.zip | |
[SPARK-8357] Fix unsafe memory leak on empty inputs in GeneratedAggregate
This patch fixes a managed memory leak in GeneratedAggregate. The leak occurs when the unsafe aggregation path is used to perform grouped aggregation on an empty input; in this case, GeneratedAggregate allocates an UnsafeFixedWidthAggregationMap that is never cleaned up because `next()` is never called on the aggregate result iterator.
This patch fixes this by short-circuiting on empty inputs.
This patch is an updated version of #6810.
Closes #6810.
Author: navis.ryu <navis@apache.org>
Author: Josh Rosen <joshrosen@databricks.com>
Closes #7560 from JoshRosen/SPARK-8357 and squashes the following commits:
3486ce4 [Josh Rosen] Some minor cleanup
c649310 [Josh Rosen] Revert SparkPlan change:
3c7db0f [Josh Rosen] Merge remote-tracking branch 'origin/master' into SPARK-8357
adc8239 [Josh Rosen] Back out Projection changes.
c5419b3 [navis.ryu] addressed comments
143e1ef [navis.ryu] fixed format & added test for CCE case
735972f [navis.ryu] used new conf apis
1a02a55 [navis.ryu] Rolled-back test-conf cleanup & fixed possible CCE & added more tests
51178e8 [navis.ryu] addressed comments
4d326b9 [navis.ryu] fixed test fails
15c5afc [navis.ryu] added a test as suggested by JoshRosen
d396589 [navis.ryu] added comments
1b07556 [navis.ryu] [SPARK-8357] [SQL] Memory leakage on unsafe aggregation path with empty input
Diffstat (limited to 'project')
0 files changed, 0 insertions, 0 deletions