diff options
| author | Michael Armbrust <michael@databricks.com> | 2014-08-13 17:40:59 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-08-13 17:40:59 -0700 |
| commit | 9fde1ff5fc114b5edb755ed40944607419b62184 (patch) | |
| tree | 079a7bc640acba973e2705c8b0d8b150cc385144 /sql/core/src/main | |
| parent | 376a82e196e102ef49b9722e8be0b01ac5890a8b (diff) | |
| download | spark-9fde1ff5fc114b5edb755ed40944607419b62184.tar.gz spark-9fde1ff5fc114b5edb755ed40944607419b62184.tar.bz2 spark-9fde1ff5fc114b5edb755ed40944607419b62184.zip | |
[SPARK-2935][SQL]Fix parquet predicate push down bug
Author: Michael Armbrust <michael@databricks.com>
Closes #1863 from marmbrus/parquetPredicates and squashes the following commits:
10ad202 [Michael Armbrust] left <=> right
f249158 [Michael Armbrust] quiet parquet tests.
802da5b [Michael Armbrust] Add test case.
eab2eda [Michael Armbrust] Fix parquet predicate push down bug
Diffstat (limited to 'sql/core/src/main')
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala b/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala index cc575bedd8..2298a9b933 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetFilters.scala @@ -201,8 +201,9 @@ object ParquetFilters { (leftFilter, rightFilter) match { case (None, Some(filter)) => Some(filter) case (Some(filter), None) => Some(filter) - case (_, _) => - Some(new AndFilter(leftFilter.get, rightFilter.get)) + case (Some(leftF), Some(rightF)) => + Some(new AndFilter(leftF, rightF)) + case _ => None } } case p @ EqualTo(left: Literal, right: NamedExpression) if !right.nullable => |