diff options
| author | windpiger <songjun@outlook.com> | 2017-01-27 17:17:17 -0800 |
|---|---|---|
| committer | gatorsmile <gatorsmile@gmail.com> | 2017-01-27 17:17:17 -0800 |
| commit | 1b5ee2003c368d18a5f8c17c2a869ef5770c60a1 (patch) | |
| tree | d73dcb492c9e0b14e868a68eff748a498af5ecc9 /LICENSE | |
| parent | bb1a1fe05e293c480c88123d4c83a6b8c25f6e2e (diff) | |
| download | spark-1b5ee2003c368d18a5f8c17c2a869ef5770c60a1.tar.gz spark-1b5ee2003c368d18a5f8c17c2a869ef5770c60a1.tar.bz2 spark-1b5ee2003c368d18a5f8c17c2a869ef5770c60a1.zip | |
[SPARK-19359][SQL] clear useless path after rename a partition with upper-case by HiveExternalCatalog
## What changes were proposed in this pull request?
Hive metastore is not case preserving and keep partition columns with lower case names.
If SparkSQL create a table with upper-case partion name use HiveExternalCatalog, when we rename partition, it first call the HiveClient to renamePartition, which will create a new lower case partition path, then SparkSql rename the lower case path to the upper-case.
while if the renamed partition contains more than one depth partition ,e.g. A=1/B=2, hive renamePartition change to a=1/b=2, then SparkSql rename it to A=1/B=2, but the a=1 still exists in the filesystem, we should also delete it.
## How was this patch tested?
unit test added
Author: windpiger <songjun@outlook.com>
Closes #16700 from windpiger/clearUselessPathAfterRenamPartition.
Diffstat (limited to 'LICENSE')
0 files changed, 0 insertions, 0 deletions