diff options
| author | Tejas Patil <tejasp@fb.com> | 2017-03-12 20:08:44 -0700 |
|---|---|---|
| committer | Xiao Li <gatorsmile@gmail.com> | 2017-03-12 20:08:44 -0700 |
| commit | 9456688547522a62f1e7520e9b3564550c57aa5d (patch) | |
| tree | 6c4411d2b0edc9ac54dbc00e9e657e73d1265eaa /sql/core/src | |
| parent | 0a4d06a7c3db9fec2b6f050a631e8b59b0e9376e (diff) | |
| download | spark-9456688547522a62f1e7520e9b3564550c57aa5d.tar.gz spark-9456688547522a62f1e7520e9b3564550c57aa5d.tar.bz2 spark-9456688547522a62f1e7520e9b3564550c57aa5d.zip | |
[SPARK-17495][SQL] Support date, timestamp and interval types in Hive hash
## What changes were proposed in this pull request?
- Timestamp hashing is done as per [TimestampWritable.hashCode()](https://github.com/apache/hive/blob/ff67cdda1c538dc65087878eeba3e165cf3230f4/serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java#L406) in Hive
- Interval hashing is done as per [HiveIntervalDayTime.hashCode()](https://github.com/apache/hive/blob/ff67cdda1c538dc65087878eeba3e165cf3230f4/storage-api/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java#L178). Note that there are inherent differences in how Hive and Spark store intervals under the hood which limits the ability to be in completely sync with hive's hashing function. I have explained this in the method doc.
- Date type was already supported. This PR adds test for that.
## How was this patch tested?
Added unit tests
Author: Tejas Patil <tejasp@fb.com>
Closes #17062 from tejasapatil/SPARK-17495_time_related_types.
Diffstat (limited to 'sql/core/src')
0 files changed, 0 insertions, 0 deletions