diff options
| author | Cheng Hao <hao.cheng@intel.com> | 2014-11-20 16:50:59 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-11-20 16:50:59 -0800 |
| commit | 84d79ee9ec47465269f7b0a7971176da93c96f3f (patch) | |
| tree | ef29e8beb3255b5eb5be01a335d5d422ef466100 /sql/hive/src/test | |
| parent | d39f2e9c683a4ab78b29eb3c5668325bf8568e8c (diff) | |
| download | spark-84d79ee9ec47465269f7b0a7971176da93c96f3f.tar.gz spark-84d79ee9ec47465269f7b0a7971176da93c96f3f.tar.bz2 spark-84d79ee9ec47465269f7b0a7971176da93c96f3f.zip | |
[SPARK-4244] [SQL] Support Hive Generic UDFs with constant object inspector parameters
Query `SELECT named_struct(lower("AA"), "12", lower("Bb"), "13") FROM src LIMIT 1` will throw exception, some of the Hive Generic UDF/UDAF requires the input object inspector is `ConstantObjectInspector`, however, we won't get that before the expression optimization executed. (Constant Folding).
This PR is a work around to fix this. (As ideally, the `output` of LogicalPlan should be identical before and after Optimization).
Author: Cheng Hao <hao.cheng@intel.com>
Closes #3109 from chenghao-intel/optimized and squashes the following commits:
487ff79 [Cheng Hao] rebase to the latest master & update the unittest
Diffstat (limited to 'sql/hive/src/test')
2 files changed, 9 insertions, 0 deletions
diff --git a/sql/hive/src/test/resources/golden/constant object inspector for generic udf-0-cc120a2331158f570a073599985d3f55 b/sql/hive/src/test/resources/golden/constant object inspector for generic udf-0-cc120a2331158f570a073599985d3f55 new file mode 100644 index 0000000000..7bc77e7f2a --- /dev/null +++ b/sql/hive/src/test/resources/golden/constant object inspector for generic udf-0-cc120a2331158f570a073599985d3f55 @@ -0,0 +1 @@ +{"aa":"10","aaaaaa":"11","aaaaaa":"12","bb12":"13","s14s14":"14"} diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala index 0dd766f253..af45dfd6e2 100644 --- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala +++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala @@ -56,6 +56,14 @@ class HiveQuerySuite extends HiveComparisonTest with BeforeAndAfter { Locale.setDefault(originalLocale) } + createQueryTest("constant object inspector for generic udf", + """SELECT named_struct( + lower("AA"), "10", + repeat(lower("AA"), 3), "11", + lower(repeat("AA", 3)), "12", + printf("Bb%d", 12), "13", + repeat(printf("s%d", 14), 2), "14") FROM src LIMIT 1""") + createQueryTest("NaN to Decimal", "SELECT CAST(CAST('NaN' AS DOUBLE) AS DECIMAL(1,1)) FROM src LIMIT 1") |