diff options
author | Cheng Hao <hao.cheng@intel.com> | 2014-12-17 15:01:59 -0800 |
---|---|---|
committer | Michael Armbrust <michael@databricks.com> | 2014-12-17 15:01:59 -0800 |
commit | 8d0d2a65eb3a7b1865f7fa7cc18b146fc6474620 (patch) | |
tree | 7acd7d206f9bfa0e600775c98d56ed98501261ea /sbt | |
parent | 19c0faad6d04349952ec25ce5ae94b718d7e8518 (diff) | |
download | spark-8d0d2a65eb3a7b1865f7fa7cc18b146fc6474620.tar.gz spark-8d0d2a65eb3a7b1865f7fa7cc18b146fc6474620.tar.bz2 spark-8d0d2a65eb3a7b1865f7fa7cc18b146fc6474620.zip |
[SPARK-4856] [SQL] NullType instead of StringType when sampling against empty string or nul...
```
TestSQLContext.sparkContext.parallelize(
"""{"ip":"27.31.100.29","headers":{"Host":"1.abc.com","Charset":"UTF-8"}}""" ::
"""{"ip":"27.31.100.29","headers":{}}""" ::
"""{"ip":"27.31.100.29","headers":""}""" :: Nil)
```
As empty string (the "headers") will be considered as String in the beginning (in line 2 and 3), it ignores the real nested data type (struct type "headers" in line 1), and also take the line 1 (the "headers") as String Type, which is not our expected.
Author: Cheng Hao <hao.cheng@intel.com>
Closes #3708 from chenghao-intel/json and squashes the following commits:
e7a72e9 [Cheng Hao] add more concise unit test
853de51 [Cheng Hao] NullType instead of StringType when sampling against empty string or null value
Diffstat (limited to 'sbt')
0 files changed, 0 insertions, 0 deletions