diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-10-25 15:00:33 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-10-25 15:00:33 +0800 |
| commit | 6f31833dbe0b766dfe4540a240fe92ebb7e14737 (patch) | |
| tree | 4547b60b8fc57abe061ecc790bc71e86bb1e7cb1 /mllib/src | |
| parent | 78d740a08a04b74b49b5cba4bb6a821631390ab4 (diff) | |
| download | spark-6f31833dbe0b766dfe4540a240fe92ebb7e14737.tar.gz spark-6f31833dbe0b766dfe4540a240fe92ebb7e14737.tar.bz2 spark-6f31833dbe0b766dfe4540a240fe92ebb7e14737.zip | |
[SPARK-18026][SQL] should not always lowercase partition columns of partition spec in parser
## What changes were proposed in this pull request?
Currently we always lowercase the partition columns of partition spec in parser, with the assumption that table partition columns are always lowercased.
However, this is not true for data source tables, which are case preserving. It's safe for now because data source tables don't store partition spec in metastore and don't support `ADD PARTITION`, `DROP PARTITION`, `RENAME PARTITION`, but we should make our code future-proof.
This PR makes partition spec case preserving at parser, and improve the `PreprocessTableInsertion` analyzer rule to normalize the partition columns in partition spec, w.r.t. the table partition columns.
## How was this patch tested?
existing tests.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #15566 from cloud-fan/partition-spec.
Diffstat (limited to 'mllib/src')
0 files changed, 0 insertions, 0 deletions