diff options
author | gatorsmile <gatorsmile@gmail.com> | 2017-02-08 10:11:44 -0500 |
---|---|---|
committer | gatorsmile <gatorsmile@gmail.com> | 2017-02-08 10:11:44 -0500 |
commit | 4d4d0de7f64cefbca28dc532b7864de9626aa241 (patch) | |
tree | cb18df849f06b90cdaae81c0a89ab0f55c4edb41 /sql/catalyst | |
parent | 0077bfcb93832d93009f73f4b80f2e3d98fd2fa4 (diff) | |
download | spark-4d4d0de7f64cefbca28dc532b7864de9626aa241.tar.gz spark-4d4d0de7f64cefbca28dc532b7864de9626aa241.tar.bz2 spark-4d4d0de7f64cefbca28dc532b7864de9626aa241.zip |
[SPARK-19279][SQL][FOLLOW-UP] Infer Schema for Hive Serde Tables
### What changes were proposed in this pull request?
`table.schema` is always not empty for partitioned tables, because `table.schema` also contains the partitioned columns, even if the original table does not have any column. This PR is to fix the issue.
### How was this patch tested?
Added a test case
Author: gatorsmile <gatorsmile@gmail.com>
Closes #16848 from gatorsmile/inferHiveSerdeSchema.
Diffstat (limited to 'sql/catalyst')
-rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala index a8fa78d41c..353e5954fd 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala @@ -194,6 +194,14 @@ case class CatalogTable( StructType(partitionFields) } + /** + * schema of this table's data columns + */ + def dataSchema: StructType = { + val dataFields = schema.dropRight(partitionColumnNames.length) + StructType(dataFields) + } + /** Return the database this table was specified to belong to, assuming it exists. */ def database: String = identifier.database.getOrElse { throw new AnalysisException(s"table $identifier did not specify database") |