diff options
author | windpiger <songjun@outlook.com> | 2017-01-24 20:49:23 +0800 |
---|---|---|
committer | Wenchen Fan <wenchen@databricks.com> | 2017-01-24 20:49:23 +0800 |
commit | 752502be053c66a95b04204b4ae0e9574394bc58 (patch) | |
tree | 1657aada20a0b994b24d74b5141ef81e860055c1 /sql/catalyst | |
parent | 3c86fdddf4bb1eac985654f80c3c716b7ae7540b (diff) | |
download | spark-752502be053c66a95b04204b4ae0e9574394bc58.tar.gz spark-752502be053c66a95b04204b4ae0e9574394bc58.tar.bz2 spark-752502be053c66a95b04204b4ae0e9574394bc58.zip |
[SPARK-19246][SQL] CataLogTable's partitionSchema order and exist check
## What changes were proposed in this pull request?
CataLogTable's partitionSchema should check if each column name in partitionColumnNames must match one and only one field in schema, if not we should throw an exception
and CataLogTable's partitionSchema should keep order with partitionColumnNames
## How was this patch tested?
N/A
Author: windpiger <songjun@outlook.com>
Closes #16606 from windpiger/checkPartionColNameWithSchema.
Diffstat (limited to 'sql/catalyst')
-rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala | 13 |
1 files changed, 9 insertions, 4 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala index 7bbaf6eb94..b8dc5f9590 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala @@ -182,10 +182,15 @@ case class CatalogTable( import CatalogTable._ - /** schema of this table's partition columns */ - def partitionSchema: StructType = StructType(schema.filter { - c => partitionColumnNames.contains(c.name) - }) + /** + * schema of this table's partition columns + */ + def partitionSchema: StructType = { + val partitionFields = schema.takeRight(partitionColumnNames.length) + assert(partitionFields.map(_.name) == partitionColumnNames) + + StructType(partitionFields) + } /** Return the database this table was specified to belong to, assuming it exists. */ def database: String = identifier.database.getOrElse { |