diff options
| author | Davies Liu <davies@databricks.com> | 2016-04-06 15:33:39 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-04-06 15:33:39 -0700 |
| commit | 5a4b11a901703464b9261dea0642d80cf8d4856c (patch) | |
| tree | 8807e70f4508b8e6253ebc116b9ea2b1475997b6 /scalastyle-config.xml | |
| parent | a4ead6d3881f071a2ae53ff1c961c6ac388cac1d (diff) | |
| download | spark-5a4b11a901703464b9261dea0642d80cf8d4856c.tar.gz spark-5a4b11a901703464b9261dea0642d80cf8d4856c.tar.bz2 spark-5a4b11a901703464b9261dea0642d80cf8d4856c.zip | |
[SPARK-14224] [SPARK-14223] [SPARK-14310] [SQL] fix RowEncoder and parquet reader for wide table
## What changes were proposed in this pull request?
1) fix the RowEncoder for wide table (many columns) by splitting the generate code into multiple functions.
2) Separate DataSourceScan as RowDataSourceScan and BatchedDataSourceScan
3) Disable the returning columnar batch in parquet reader if there are many columns.
4) Added a internal config for maximum number of fields (nested) columns supported by whole stage codegen.
Closes #12098
## How was this patch tested?
Add a tests for table with 1000 columns.
Author: Davies Liu <davies@databricks.com>
Closes #12047 from davies/many_columns.
Diffstat (limited to 'scalastyle-config.xml')
0 files changed, 0 insertions, 0 deletions