diff options
| author | Ryan Blue <blue@apache.org> | 2016-05-09 17:01:23 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-05-09 17:01:23 +0800 |
| commit | 652bbb1bf62722b08a062c7a2bf72019f85e179e (patch) | |
| tree | 5e4d588f2c9070a0c645c176b7bd42dd34fe04b5 /examples/src | |
| parent | 16a503cf0af3e7c703d56a1a730e4f3a534f6b3c (diff) | |
| download | spark-652bbb1bf62722b08a062c7a2bf72019f85e179e.tar.gz spark-652bbb1bf62722b08a062c7a2bf72019f85e179e.tar.bz2 spark-652bbb1bf62722b08a062c7a2bf72019f85e179e.zip | |
[SPARK-14459][SQL] Detect relation partitioning and adjust the logical plan
## What changes were proposed in this pull request?
This detects a relation's partitioning and adds checks to the analyzer.
If an InsertIntoTable node has no partitioning, it is replaced by the
relation's partition scheme and input columns are correctly adjusted,
placing the partition columns at the end in partition order. If an
InsertIntoTable node has partitioning, it is checked against the table's
reported partitions.
These changes required adding a PartitionedRelation trait to the catalog
interface because Hive's MetastoreRelation doesn't extend
CatalogRelation.
This commit also includes a fix to InsertIntoTable's resolved logic,
which now detects that all expected columns are present, including
dynamic partition columns. Previously, the number of expected columns
was not checked and resolved was true if there were missing columns.
## How was this patch tested?
This adds new tests to the InsertIntoTableSuite that are fixed by this PR.
Author: Ryan Blue <blue@apache.org>
Closes #12239 from rdblue/SPARK-14459-detect-hive-partitioning.
Diffstat (limited to 'examples/src')
0 files changed, 0 insertions, 0 deletions