diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-05-11 23:55:42 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2016-05-11 23:55:42 -0700 |
| commit | 46991448aa6f78f413a761059d7d7bb586f9d63e (patch) | |
| tree | 1fceadbdb3eb6d38103e1d6dd029a21d12c8cb41 /sql/hive/src/main/scala | |
| parent | 9e266d07a444fd465fe178cdd5c4894cd09cbda3 (diff) | |
| download | spark-46991448aa6f78f413a761059d7d7bb586f9d63e.tar.gz spark-46991448aa6f78f413a761059d7d7bb586f9d63e.tar.bz2 spark-46991448aa6f78f413a761059d7d7bb586f9d63e.zip | |
[SPARK-15160][SQL] support data source table in InMemoryCatalog
## What changes were proposed in this pull request?
This PR adds a new rule to convert `SimpleCatalogRelation` to data source table if its table property contains data source information.
## How was this patch tested?
new test in SQLQuerySuite
Author: Wenchen Fan <wenchen@databricks.com>
Closes #12935 from cloud-fan/ds-table.
Diffstat (limited to 'sql/hive/src/main/scala')
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala index 8cfcec79cd..2f20cde4b1 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala @@ -71,6 +71,8 @@ private[hive] class HiveMetastoreCatalog(sparkSession: SparkSession) extends Log logDebug(s"Creating new cached data source for $in") val table = client.getTable(in.database, in.name) + // TODO: the following code is duplicated with FindDataSourceTable.readDataSourceTable + def schemaStringFromParts: Option[String] = { table.properties.get("spark.sql.sources.schema.numParts").map { numParts => val parts = (0 until numParts.toInt).map { index => |