diff options
| author | Michael Armbrust <michael@databricks.com> | 2015-06-19 16:54:51 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2015-06-19 16:54:51 -0700 |
| commit | a333a72e029d2546a66b36d6b3458e965430c530 (patch) | |
| tree | 417bd90555d96869f11357af7c373be50e206b53 /external | |
| parent | 9814b971f07dff8a99f1b8ad2adf70614f1c690b (diff) | |
| download | spark-a333a72e029d2546a66b36d6b3458e965430c530.tar.gz spark-a333a72e029d2546a66b36d6b3458e965430c530.tar.bz2 spark-a333a72e029d2546a66b36d6b3458e965430c530.zip | |
[SPARK-8420] [SQL] Fix comparision of timestamps/dates with strings
In earlier versions of Spark SQL we casted `TimestampType` and `DataType` to `StringType` when it was involved in a binary comparison with a `StringType`. This allowed comparing a timestamp with a partial date as a user would expect.
- `time > "2014-06-10"`
- `time > "2014"`
In 1.4.0 we tried to cast the String instead into a Timestamp. However, since partial dates are not a valid complete timestamp this results in `null` which results in the tuple being filtered.
This PR restores the earlier behavior. Note that we still special case equality so that these comparisons are not affected by not printing zeros for subsecond precision.
Author: Michael Armbrust <michael@databricks.com>
Closes #6888 from marmbrus/timeCompareString and squashes the following commits:
bdef29c [Michael Armbrust] test partial date
1f09adf [Michael Armbrust] special handling of equality
1172c60 [Michael Armbrust] more test fixing
4dfc412 [Michael Armbrust] fix tests
aaa9508 [Michael Armbrust] newline
04d908f [Michael Armbrust] [SPARK-8420][SQL] Fix comparision of timestamps/dates with strings
Diffstat (limited to 'external')
0 files changed, 0 insertions, 0 deletions