diff options
| author | Herman van Hovell <hvanhovell@databricks.com> | 2016-11-28 07:10:52 -0800 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-11-28 07:10:52 -0800 |
| commit | 38e29824d9a50464daa397c28e89610ed0aed4b6 (patch) | |
| tree | 91de58c8327d60437eb6ba3ca2064d6f0e388c52 /dev | |
| parent | 9f273c5173c05017c3009faaf3e10f2f70a842d0 (diff) | |
| download | spark-38e29824d9a50464daa397c28e89610ed0aed4b6.tar.gz spark-38e29824d9a50464daa397c28e89610ed0aed4b6.tar.bz2 spark-38e29824d9a50464daa397c28e89610ed0aed4b6.zip | |
[SPARK-18597][SQL] Do not push-down join conditions to the right side of a LEFT ANTI join
## What changes were proposed in this pull request?
We currently push down join conditions of a Left Anti join to both sides of the join. This is similar to Inner, Left Semi and Existence (a specialized left semi) join. The problem is that this changes the semantics of the join; a left anti join filters out rows that matches the join condition.
This PR fixes this by only pushing down conditions to the left hand side of the join. This is similar to the behavior of left outer join.
## How was this patch tested?
Added tests to `FilterPushdownSuite.scala` and created a SQLQueryTestSuite file for left anti joins with a regression test.
Author: Herman van Hovell <hvanhovell@databricks.com>
Closes #16026 from hvanhovell/SPARK-18597.
Diffstat (limited to 'dev')
0 files changed, 0 insertions, 0 deletions