diff options
| author | Yin Huai <huaiyin.thu@gmail.com> | 2014-07-07 17:05:59 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-07-07 17:05:59 -0700 |
| commit | f0496ee10847db921a028a34f70385f9b740b3f3 (patch) | |
| tree | e7ce39f5c6c6c7ad11fe7a904ee488682430e387 /python/pyspark | |
| parent | 4deeed17c4847f212a4fa1a8685cfe8a12179263 (diff) | |
| download | spark-f0496ee10847db921a028a34f70385f9b740b3f3.tar.gz spark-f0496ee10847db921a028a34f70385f9b740b3f3.tar.bz2 spark-f0496ee10847db921a028a34f70385f9b740b3f3.zip | |
[SPARK-2375][SQL] JSON schema inference may not resolve type conflicts correctly for a field inside an array of structs
For example, for
```
{"array": [{"field":214748364700}, {"field":1}]}
```
the type of field is resolved as IntType. While, for
```
{"array": [{"field":1}, {"field":214748364700}]}
```
the type of field is resolved as LongType.
JIRA: https://issues.apache.org/jira/browse/SPARK-2375
Author: Yin Huai <huaiyin.thu@gmail.com>
Closes #1308 from yhuai/SPARK-2375 and squashes the following commits:
3e2e312 [Yin Huai] Update unit test.
1b2ff9f [Yin Huai] Merge remote-tracking branch 'upstream/master' into SPARK-2375
10794eb [Yin Huai] Correctly resolve the type of a field inside an array of structs.
Diffstat (limited to 'python/pyspark')
0 files changed, 0 insertions, 0 deletions