aboutsummaryrefslogtreecommitdiff
path: root/sql/catalyst/src/test
diff options
context:
space:
mode:
authorSean Zhong <seanzhong@databricks.com>2016-08-25 16:36:16 -0700
committerYin Huai <yhuai@databricks.com>2016-08-25 16:36:16 -0700
commitd96d1515638da20b594f7bfe3cfdb50088f25a04 (patch)
tree69e7803b4f49d0ed03073795843eb95d8f63529f /sql/catalyst/src/test
parent9b5a1d1d53bc4412de3cbc86dc819b0c213229a8 (diff)
downloadspark-d96d1515638da20b594f7bfe3cfdb50088f25a04.tar.gz
spark-d96d1515638da20b594f7bfe3cfdb50088f25a04.tar.bz2
spark-d96d1515638da20b594f7bfe3cfdb50088f25a04.zip
[SPARK-17187][SQL] Supports using arbitrary Java object as internal aggregation buffer object
## What changes were proposed in this pull request? This PR introduces an abstract class `TypedImperativeAggregate` so that an aggregation function of TypedImperativeAggregate can use **arbitrary** user-defined Java object as intermediate aggregation buffer object. **This has advantages like:** 1. It now can support larger category of aggregation functions. For example, it will be much easier to implement aggregation function `percentile_approx`, which has a complex aggregation buffer definition. 2. It can be used to avoid doing serialization/de-serialization for every call of `update` or `merge` when converting domain specific aggregation object to internal Spark-Sql storage format. 3. It is easier to integrate with other existing monoid libraries like algebird, and supports more aggregation functions with high performance. Please see `org.apache.spark.sql.TypedImperativeAggregateSuite.TypedMaxAggregate` to find an example of how to defined a `TypedImperativeAggregate` aggregation function. Please see Java doc of `TypedImperativeAggregate` and Jira ticket SPARK-17187 for more information. ## How was this patch tested? Unit tests. Author: Sean Zhong <seanzhong@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #14753 from clockfly/object_aggregation_buffer_try_2.
Diffstat (limited to 'sql/catalyst/src/test')
0 files changed, 0 insertions, 0 deletions