aboutsummaryrefslogtreecommitdiff
path: root/R/pkg/NAMESPACE
diff options
context:
space:
mode:
authorCheng Lian <lian@databricks.com>2015-08-16 10:17:58 -0700
committerYin Huai <yhuai@databricks.com>2015-08-16 10:17:58 -0700
commitae2370e72f93db8a28b262e8252c55fe1fc9873c (patch)
treeb3bf8b6699430bfd4f0b2ecef0103d40bf1d3f76 /R/pkg/NAMESPACE
parentcf016075a006034c24c5b758edb279f3e151d25d (diff)
downloadspark-ae2370e72f93db8a28b262e8252c55fe1fc9873c.tar.gz
spark-ae2370e72f93db8a28b262e8252c55fe1fc9873c.tar.bz2
spark-ae2370e72f93db8a28b262e8252c55fe1fc9873c.zip
[SPARK-10005] [SQL] Fixes schema merging for nested structs
In case of schema merging, we only handled first level fields when converting Parquet groups to `InternalRow`s. Nested struct fields are not properly handled. For example, the schema of a Parquet file to be read can be: ``` message individual { required group f1 { optional binary f11 (utf8); } } ``` while the global schema is: ``` message global { required group f1 { optional binary f11 (utf8); optional int32 f12; } } ``` This PR fixes this issue by padding missing fields when creating actual converters. Author: Cheng Lian <lian@databricks.com> Closes #8228 from liancheng/spark-10005/nested-schema-merging.
Diffstat (limited to 'R/pkg/NAMESPACE')
0 files changed, 0 insertions, 0 deletions