diff options
author | Wenchen Fan <wenchen@databricks.com> | 2016-11-23 04:15:19 -0800 |
---|---|---|
committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-11-23 04:15:19 -0800 |
commit | 84284e8c82542d80dad94e458a0c0210bf803db3 (patch) | |
tree | 55d389c2a6b79350c09687ae04c061515aa12285 /sql/core | |
parent | 85235ed6c600270e3fa434738bd50dce3564440a (diff) | |
download | spark-84284e8c82542d80dad94e458a0c0210bf803db3.tar.gz spark-84284e8c82542d80dad94e458a0c0210bf803db3.tar.bz2 spark-84284e8c82542d80dad94e458a0c0210bf803db3.zip |
[SPARK-18053][SQL] compare unsafe and safe complex-type values correctly
## What changes were proposed in this pull request?
In Spark SQL, some expression may output safe format values, e.g. `CreateArray`, `CreateStruct`, `Cast`, etc. When we compare 2 values, we should be able to compare safe and unsafe formats.
The `GreaterThan`, `LessThan`, etc. in Spark SQL already handles it, but the `EqualTo` doesn't. This PR fixes it.
## How was this patch tested?
new unit test and regression test
Author: Wenchen Fan <wenchen@databricks.com>
Closes #15929 from cloud-fan/type-aware.
Diffstat (limited to 'sql/core')
-rw-r--r-- | sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala index a715176d55..d2ec3cfc05 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala @@ -2469,4 +2469,11 @@ class SQLQuerySuite extends QueryTest with SharedSQLContext { } } } + + test("SPARK-18053: ARRAY equality is broken") { + withTable("array_tbl") { + spark.range(10).select(array($"id").as("arr")).write.saveAsTable("array_tbl") + assert(sql("SELECT * FROM array_tbl where arr = ARRAY(1L)").count == 1) + } + } } |