diff options
| author | Brian Cho <bcho@fb.com> | 2016-06-22 10:38:42 -0700 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-06-22 10:38:42 -0700 |
| commit | 0a9c02759515c41de37db6381750bc3a316c860c (patch) | |
| tree | e45257d175300ab41cd1bc2ea2289de65372d6df /sql/hive/src/test | |
| parent | 6a6010f0015542dc2753b2cb12fdd1204db63ea6 (diff) | |
| download | spark-0a9c02759515c41de37db6381750bc3a316c860c.tar.gz spark-0a9c02759515c41de37db6381750bc3a316c860c.tar.bz2 spark-0a9c02759515c41de37db6381750bc3a316c860c.zip | |
[SPARK-15956][SQL] When unwrapping ORC avoid pattern matching at runtime
## What changes were proposed in this pull request?
Extend the returning of unwrapper functions from primitive types to all types.
## How was this patch tested?
The patch should pass all unit tests. Reading ORC files with non-primitive types with this change reduced the read time by ~15%.
===
The github diff is very noisy. Attaching the screenshots below for improved readability:
Author: Brian Cho <bcho@fb.com>
Closes #13676 from dafrista/improve-orc-master.
Diffstat (limited to 'sql/hive/src/test')
| -rw-r--r-- | sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala index 3b867bbfa1..bc51bcb07e 100644 --- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala +++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala @@ -35,6 +35,12 @@ import org.apache.spark.sql.types._ import org.apache.spark.sql.Row class HiveInspectorSuite extends SparkFunSuite with HiveInspectors { + + def unwrap(data: Any, oi: ObjectInspector): Any = { + val unwrapper = unwrapperFor(oi) + unwrapper(data) + } + test("Test wrap SettableStructObjectInspector") { val udaf = new UDAFPercentile.PercentileLongEvaluator() udaf.init() |