diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2017-01-21 13:57:50 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2017-01-21 13:57:50 +0800 |
| commit | 3c2ba9fcc493504c9e7d3caf0b93256ca299cbfe (patch) | |
| tree | 6ed86e1626f75b5672c9ba451db059dcde664a55 /core/src/main | |
| parent | f174cdc7478d0b81f9cfa896284a5ec4c6bb952d (diff) | |
| download | spark-3c2ba9fcc493504c9e7d3caf0b93256ca299cbfe.tar.gz spark-3c2ba9fcc493504c9e7d3caf0b93256ca299cbfe.tar.bz2 spark-3c2ba9fcc493504c9e7d3caf0b93256ca299cbfe.zip | |
[SPARK-19305][SQL] partitioned table should always put partition columns at the end of table schema
## What changes were proposed in this pull request?
For data source tables, we will always reorder the specified table schema, or the query in CTAS, to put partition columns at the end. e.g. `CREATE TABLE t(a int, b int, c int, d int) USING parquet PARTITIONED BY (d, b)` will create a table with schema `<a, c, d, b>`
Hive serde tables don't have this problem before, because its CREATE TABLE syntax specifies data schema and partition schema individually.
However, after we unifed the CREATE TABLE syntax, Hive serde table also need to do the reorder. This PR puts the reorder logic in a analyzer rule, which works with both data source tables and Hive serde tables.
## How was this patch tested?
new regression test
Author: Wenchen Fan <wenchen@databricks.com>
Closes #16655 from cloud-fan/schema.
Diffstat (limited to 'core/src/main')
0 files changed, 0 insertions, 0 deletions