diff options
| author | Dongjoon Hyun <dongjoon@apache.org> | 2016-04-18 12:26:56 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-04-18 12:26:56 -0700 |
| commit | d280d1da1aec925687a0bfb496f3a6e0979e896f (patch) | |
| tree | dbeb76ebe45f519f68b624e0cda78d3e719d2dcc /python | |
| parent | b64482f49f6b9c7ff0ba64bd3202fe9cc6ad119a (diff) | |
| download | spark-d280d1da1aec925687a0bfb496f3a6e0979e896f.tar.gz spark-d280d1da1aec925687a0bfb496f3a6e0979e896f.tar.bz2 spark-d280d1da1aec925687a0bfb496f3a6e0979e896f.zip | |
[SPARK-14580][SPARK-14655][SQL] Hive IfCoercion should preserve predicate.
## What changes were proposed in this pull request?
Currently, `HiveTypeCoercion.IfCoercion` removes all predicates whose return-type are null. However, some UDFs need evaluations because they are designed to throw exceptions. This PR fixes that to preserve the predicates. Also, `assert_true` is implemented as Spark SQL function.
**Before**
```
scala> sql("select if(assert_true(false),2,3)").head
res2: org.apache.spark.sql.Row = [3]
```
**After**
```
scala> sql("select if(assert_true(false),2,3)").head
... ASSERT_TRUE ...
```
**Hive**
```
hive> select if(assert_true(false),2,3);
OK
Failed with exception java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: ASSERT_TRUE(): assertion failed.
```
## How was this patch tested?
Pass the Jenkins tests (including a new testcase in `HivePlanTest`)
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #12340 from dongjoon-hyun/SPARK-14580.
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions