diff options
| author | Herman van Hovell <hvanhovell@databricks.com> | 2016-09-03 19:02:20 +0200 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-09-03 19:02:20 +0200 |
| commit | c2a1576c230697f56f282b6388c79835377e0f2f (patch) | |
| tree | 9d870d3a6366c728a9058d67c3bb1bb123859828 /R/pkg | |
| parent | a8a35b39b92fc9000eaac102c67c66be30b05e54 (diff) | |
| download | spark-c2a1576c230697f56f282b6388c79835377e0f2f.tar.gz spark-c2a1576c230697f56f282b6388c79835377e0f2f.tar.bz2 spark-c2a1576c230697f56f282b6388c79835377e0f2f.zip | |
[SPARK-17335][SQL] Fix ArrayType and MapType CatalogString.
## What changes were proposed in this pull request?
the `catalogString` for `ArrayType` and `MapType` currently calls the `simpleString` method on its children. This is a problem when the child is a struct, the `struct.simpleString` implementation truncates the number of fields it shows (25 at max). This breaks the generation of a proper `catalogString`, and has shown to cause errors while writing to Hive.
This PR fixes this by providing proper `catalogString` implementations for `ArrayData` or `MapData`.
## How was this patch tested?
Added testing for `catalogString` to `DataTypeSuite`.
Author: Herman van Hovell <hvanhovell@databricks.com>
Closes #14938 from hvanhovell/SPARK-17335.
Diffstat (limited to 'R/pkg')
0 files changed, 0 insertions, 0 deletions