diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2017-01-05 14:11:05 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2017-01-05 14:11:05 +0800 |
| commit | 30345c43b7d17bb00184b60a547225bae8ee78e7 (patch) | |
| tree | f7bdd72e3f720c5e8491803f9246bb140c79baa7 /docs | |
| parent | 6873430cb5ec0096991a02d4e01266945e79ebb9 (diff) | |
| download | spark-30345c43b7d17bb00184b60a547225bae8ee78e7.tar.gz spark-30345c43b7d17bb00184b60a547225bae8ee78e7.tar.bz2 spark-30345c43b7d17bb00184b60a547225bae8ee78e7.zip | |
[SPARK-19058][SQL] fix partition related behaviors with DataFrameWriter.saveAsTable
## What changes were proposed in this pull request?
When we append data to a partitioned table with `DataFrameWriter.saveAsTable`, there are 2 issues:
1. doesn't work when the partition has custom location.
2. will recover all partitions
This PR fixes them by moving the special partition handling code from `DataSourceAnalysis` to `InsertIntoHadoopFsRelationCommand`, so that the `DataFrameWriter.saveAsTable` code path can also benefit from it.
## How was this patch tested?
newly added regression tests
Author: Wenchen Fan <wenchen@databricks.com>
Closes #16460 from cloud-fan/append.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions