diff options
author | Sameer Agarwal <sameer@databricks.com> | 2016-03-16 16:27:46 -0700 |
---|---|---|
committer | Yin Huai <yhuai@databricks.com> | 2016-03-16 16:27:46 -0700 |
commit | 77ba3021c12dc63cb7d831f964f901e0474acd96 (patch) | |
tree | b0e96ecae892b36310c86a0071d634921d5876ca /project | |
parent | f96997ba244a14c26e85a2475415a762d0c0d0a8 (diff) | |
download | spark-77ba3021c12dc63cb7d831f964f901e0474acd96.tar.gz spark-77ba3021c12dc63cb7d831f964f901e0474acd96.tar.bz2 spark-77ba3021c12dc63cb7d831f964f901e0474acd96.zip |
[SPARK-13869][SQL] Remove redundant conditions while combining filters
## What changes were proposed in this pull request?
**[I'll link it to the JIRA once ASF JIRA is back online]**
This PR modifies the existing `CombineFilters` rule to remove redundant conditions while combining individual filter predicates. For instance, queries of the form `table.where('a === 1 && 'b === 1).where('a === 1 && 'c === 1)` will now be optimized to ` table.where('a === 1 && 'b === 1 && 'c === 1)` (instead of ` table.where('a === 1 && 'a === 1 && 'b === 1 && 'c === 1)`)
## How was this patch tested?
Unit test in `FilterPushdownSuite`
Author: Sameer Agarwal <sameer@databricks.com>
Closes #11670 from sameeragarwal/combine-filters.
Diffstat (limited to 'project')
0 files changed, 0 insertions, 0 deletions