diff options
| author | Reynold Xin <rxin@databricks.com> | 2016-08-18 16:37:25 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-08-18 16:37:25 +0800 |
| commit | 1748f824101870b845dbbd118763c6885744f98a (patch) | |
| tree | 5d82b921bdad119f417c3e8c291dab2f37491f60 /sql/catalyst/src/test | |
| parent | 3e6ef2e8a435a91b6a76876e9833917e5aa0945e (diff) | |
| download | spark-1748f824101870b845dbbd118763c6885744f98a.tar.gz spark-1748f824101870b845dbbd118763c6885744f98a.tar.bz2 spark-1748f824101870b845dbbd118763c6885744f98a.zip | |
[SPARK-16391][SQL] Support partial aggregation for reduceGroups
## What changes were proposed in this pull request?
This patch introduces a new private ReduceAggregator interface that is a subclass of Aggregator. ReduceAggregator only requires a single associative and commutative reduce function. ReduceAggregator is also used to implement KeyValueGroupedDataset.reduceGroups in order to support partial aggregation.
Note that the pull request was initially done by viirya.
## How was this patch tested?
Covered by original tests for reduceGroups, as well as a new test suite for ReduceAggregator.
Author: Reynold Xin <rxin@databricks.com>
Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
Closes #14576 from rxin/reduceAggregator.
Diffstat (limited to 'sql/catalyst/src/test')
0 files changed, 0 insertions, 0 deletions