diff options
| author | Wisely Chen <wiselychen@appier.com> | 2015-07-02 09:58:12 -0700 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2015-07-02 09:58:12 -0700 |
| commit | 246265f2bb056d5e9011d3331b809471a24ff8d7 (patch) | |
| tree | 73bf1cbe265db29fda5f6e33a4ba074c3be51f90 /data | |
| parent | 1bbdf9ead9e912f60dccbb23029b7de4948ebee3 (diff) | |
| download | spark-246265f2bb056d5e9011d3331b809471a24ff8d7.tar.gz spark-246265f2bb056d5e9011d3331b809471a24ff8d7.tar.bz2 spark-246265f2bb056d5e9011d3331b809471a24ff8d7.zip | |
[SPARK-8690] [SQL] Add a setting to disable SparkSQL parquet schema merge by using datasource API
The detail problem story is in https://issues.apache.org/jira/browse/SPARK-8690
General speaking, I add a config spark.sql.parquet.mergeSchema to achieve the sqlContext.load("parquet" , Map( "path" -> "..." , "mergeSchema" -> "false" ))
It will become a simple flag and without any side affect.
Author: Wisely Chen <wiselychen@appier.com>
Closes #7070 from thegiive/SPARK8690 and squashes the following commits:
c6f3e86 [Wisely Chen] Refactor some code style and merge the test case to ParquetSchemaMergeConfigSuite
94c9307 [Wisely Chen] Remove some style problem
db8ef1b [Wisely Chen] Change config to SQLConf and add test case
b6806fb [Wisely Chen] remove text
c0edb8c [Wisely Chen] [SPARK-8690] add a config spark.sql.parquet.mergeSchema to disable datasource API schema merge feature.
Diffstat (limited to 'data')
0 files changed, 0 insertions, 0 deletions