diff options
| author | Liang-Chi Hsieh <simonh@tw.ibm.com> | 2016-06-10 18:23:59 -0700 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-06-10 18:23:59 -0700 |
| commit | bba5d7999f7b3ae9d816ea552ba9378fea1615a6 (patch) | |
| tree | bf303da6296982bd924fb057a4c0a2306d6b68fa /examples/src | |
| parent | 54f758b5fc60ecb0da6b191939a72ef5829be38c (diff) | |
| download | spark-bba5d7999f7b3ae9d816ea552ba9378fea1615a6.tar.gz spark-bba5d7999f7b3ae9d816ea552ba9378fea1615a6.tar.bz2 spark-bba5d7999f7b3ae9d816ea552ba9378fea1615a6.zip | |
[SPARK-15639][SQL] Try to push down filter at RowGroups level for parquet reader
## What changes were proposed in this pull request?
The base class `SpecificParquetRecordReaderBase` used for vectorized parquet reader will try to get pushed-down filters from the given configuration. This pushed-down filters are used for RowGroups-level filtering. However, we don't set up the filters to push down into the configuration. In other words, the filters are not actually pushed down to do RowGroups-level filtering. This patch is to fix this and tries to set up the filters for pushing down to configuration for the reader.
## How was this patch tested?
Existing tests should be passed.
Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
Closes #13371 from viirya/vectorized-reader-push-down-filter.
Diffstat (limited to 'examples/src')
0 files changed, 0 insertions, 0 deletions