diff options
| author | Kevin Yu <qyu@us.ibm.com> | 2015-12-28 11:58:33 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-12-28 11:58:33 -0800 |
| commit | fd50df413fbb3b7528cdff311cc040a6212340b9 (patch) | |
| tree | 41f430c142ae5d8a8ad485a0f51f3cf703a680ef /core | |
| parent | 8543997f2daa60dfa0509f149fab207de98145a0 (diff) | |
| download | spark-fd50df413fbb3b7528cdff311cc040a6212340b9.tar.gz spark-fd50df413fbb3b7528cdff311cc040a6212340b9.tar.bz2 spark-fd50df413fbb3b7528cdff311cc040a6212340b9.zip | |
[SPARK-12231][SQL] create a combineFilters' projection when we call buildPartitionedTableScan
Hello Michael & All:
We have some issues to submit the new codes in the other PR(#10299), so we closed that PR and open this one with the fix.
The reason for the previous failure is that the projection for the scan when there is a filter that is not pushed down (the "left-over" filter) could be different, in elements or ordering, from the original projection.
With this new codes, the approach to solve this problem is:
Insert a new Project if the "left-over" filter is nonempty and (the original projection is not empty and the projection for the scan has more than one elements which could otherwise cause different ordering in projection).
We create 3 test cases to cover the otherwise failure cases.
Author: Kevin Yu <qyu@us.ibm.com>
Closes #10388 from kevinyu98/spark-12231.
Diffstat (limited to 'core')
0 files changed, 0 insertions, 0 deletions