diff options
| author | Ergin Seyfe <eseyfe@fb.com> | 2016-10-03 23:28:39 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-10-03 23:28:39 -0700 |
| commit | d2dc8c4a162834818190ffd82894522c524ca3e5 (patch) | |
| tree | 1fb2cdf4e063dabc4124fede9c90e5c3b0d128eb | |
| parent | b1b47274bfeba17a9e4e9acebd7385289f31f6c8 (diff) | |
| download | spark-d2dc8c4a162834818190ffd82894522c524ca3e5.tar.gz spark-d2dc8c4a162834818190ffd82894522c524ca3e5.tar.bz2 spark-d2dc8c4a162834818190ffd82894522c524ca3e5.zip | |
[SPARK-17773] Input/Output] Add VoidObjectInspector
## What changes were proposed in this pull request?
Added VoidObjectInspector to the list of PrimitiveObjectInspectors
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Executing following query was failing.
select SOME_UDAF*(a.arr)
from (
select Array(null) as arr from dim_one_row
) a
After the fix, I am getting the correct output:
res0: Array[org.apache.spark.sql.Row] = Array([null])
Author: Ergin Seyfe <eseyfe@fb.com>
Closes #15337 from seyfe/add_void_object_inspector.
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala | 2 | ||||
| -rw-r--r-- | sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala | 1 |
2 files changed, 3 insertions, 0 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala index c3c4351cf5..fe34caa0a3 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala @@ -319,6 +319,8 @@ private[hive] trait HiveInspectors { withNullSafe(o => getTimestampWritable(o)) case _: TimestampObjectInspector => withNullSafe(o => DateTimeUtils.toJavaTimestamp(o.asInstanceOf[Long])) + case _: VoidObjectInspector => + (_: Any) => null // always be null for void object inspector } case soi: StandardStructObjectInspector => diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala index bc51bcb07e..3de1f4aeb7 100644 --- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala +++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala @@ -81,6 +81,7 @@ class HiveInspectorSuite extends SparkFunSuite with HiveInspectors { val data = Literal(true) :: + Literal(null) :: Literal(0.asInstanceOf[Byte]) :: Literal(0.asInstanceOf[Short]) :: Literal(0) :: |