diff options
| author | Cheng Lian <lian@databricks.com> | 2015-03-26 13:11:37 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-03-26 13:11:37 -0700 |
| commit | 71a0d40ebd37c80d8020e184366778b57c762285 (patch) | |
| tree | bb91e2ec5ef9627937a90600a5a544abb6224c2d /docs/graphx-programming-guide.md | |
| parent | 784fcd532784fcfd9bf0a1db71c9f71c469ee716 (diff) | |
| download | spark-71a0d40ebd37c80d8020e184366778b57c762285.tar.gz spark-71a0d40ebd37c80d8020e184366778b57c762285.tar.bz2 spark-71a0d40ebd37c80d8020e184366778b57c762285.zip | |
[SPARK-6554] [SQL] Don't push down predicates which reference partition column(s)
There are two cases for the new Parquet data source:
1. Partition columns exist in the Parquet data files
We don't need to push-down these predicates since partition pruning already handles them.
1. Partition columns don't exist in the Parquet data files
We can't push-down these predicates since they are considered as invalid columns by Parquet.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/5210)
<!-- Reviewable:end -->
Author: Cheng Lian <lian@databricks.com>
Closes #5210 from liancheng/spark-6554 and squashes the following commits:
4f7ec03 [Cheng Lian] Adds comments
e134ced [Cheng Lian] Don't push down predicates which reference partition column(s)
Diffstat (limited to 'docs/graphx-programming-guide.md')
0 files changed, 0 insertions, 0 deletions