diff options
| author | Andrew Ray <ray.andrew@gmail.com> | 2015-11-19 15:11:30 -0800 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2015-11-19 15:11:30 -0800 |
| commit | 37cff1b1a79cad11277612cb9bc8bc2365cf5ff2 (patch) | |
| tree | cdc83803a933b06ff3ec41c958825152f0854c51 /yarn/src | |
| parent | 01403aa97b6aaab9b86ae806b5ea9e82690a741f (diff) | |
| download | spark-37cff1b1a79cad11277612cb9bc8bc2365cf5ff2.tar.gz spark-37cff1b1a79cad11277612cb9bc8bc2365cf5ff2.tar.bz2 spark-37cff1b1a79cad11277612cb9bc8bc2365cf5ff2.zip | |
[SPARK-11275][SQL] Incorrect results when using rollup/cube
Fixes bug with grouping sets (including cube/rollup) where aggregates that included grouping expressions would return the wrong (null) result.
Also simplifies the analyzer rule a bit and leaves column pruning to the optimizer.
Added multiple unit tests to DataFrameAggregateSuite and verified it passes hive compatibility suite:
```
build/sbt -Phive -Dspark.hive.whitelist='groupby.*_grouping.*' 'test-only org.apache.spark.sql.hive.execution.HiveCompatibilitySuite'
```
This is an alternative to pr https://github.com/apache/spark/pull/9419 but I think its better as it simplifies the analyzer rule instead of adding another special case to it.
Author: Andrew Ray <ray.andrew@gmail.com>
Closes #9815 from aray/groupingset-agg-fix.
Diffstat (limited to 'yarn/src')
0 files changed, 0 insertions, 0 deletions