diff options
| author | Davies Liu <davies@databricks.com> | 2016-02-25 00:13:07 -0800 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-02-25 00:13:07 -0800 |
| commit | 07f92ef1fa090821bef9c60689bf41909d781ee7 (patch) | |
| tree | 10d8f563352dffea1a2b3363bb2659174187562a /project/MimaExcludes.scala | |
| parent | 264533b553be806b6c45457201952e83c028ec78 (diff) | |
| download | spark-07f92ef1fa090821bef9c60689bf41909d781ee7.tar.gz spark-07f92ef1fa090821bef9c60689bf41909d781ee7.tar.bz2 spark-07f92ef1fa090821bef9c60689bf41909d781ee7.zip | |
[SPARK-13376] [SPARK-13476] [SQL] improve column pruning
## What changes were proposed in this pull request?
This PR mostly rewrite the ColumnPruning rule to support most of the SQL logical plans (except those for Dataset).
This PR also fix a bug in Generate, it should always output UnsafeRow, added an regression test for that.
## How was this patch tested?
This is test by unit tests, also manually test with TPCDS Q78, which could prune all unused columns successfully, improved the performance by 78% (from 22s to 12s).
Author: Davies Liu <davies@databricks.com>
Closes #11354 from davies/fix_column_pruning.
Diffstat (limited to 'project/MimaExcludes.scala')
0 files changed, 0 insertions, 0 deletions