diff options
| author | gatorsmile <gatorsmile@gmail.com> | 2016-08-25 12:50:03 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-08-25 12:50:03 +0800 |
| commit | 4d0706d616176dc29ff3562e40cb00dd4eb9c302 (patch) | |
| tree | 19654fb381ddd754631268affdc9a4575b2ba4e6 /sql/catalyst/src/main/scala/org/apache | |
| parent | ac27557eb622a257abeb3e8551f06ebc72f87133 (diff) | |
| download | spark-4d0706d616176dc29ff3562e40cb00dd4eb9c302.tar.gz spark-4d0706d616176dc29ff3562e40cb00dd4eb9c302.tar.bz2 spark-4d0706d616176dc29ff3562e40cb00dd4eb9c302.zip | |
[SPARK-17190][SQL] Removal of HiveSharedState
### What changes were proposed in this pull request?
Since `HiveClient` is used to interact with the Hive metastore, it should be hidden in `HiveExternalCatalog`. After moving `HiveClient` into `HiveExternalCatalog`, `HiveSharedState` becomes a wrapper of `HiveExternalCatalog`. Thus, removal of `HiveSharedState` becomes straightforward. After removal of `HiveSharedState`, the reflection logic is directly applied on the choice of `ExternalCatalog` types, based on the configuration of `CATALOG_IMPLEMENTATION`.
~~`HiveClient` is also used/invoked by the other entities besides HiveExternalCatalog, we defines the following two APIs: getClient and getNewClient~~
### How was this patch tested?
The existing test cases
Author: gatorsmile <gatorsmile@gmail.com>
Closes #14757 from gatorsmile/removeHiveClient.
Diffstat (limited to 'sql/catalyst/src/main/scala/org/apache')
| -rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala | 8 |
1 files changed, 6 insertions, 2 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala index 9ebf7de1a5..b55ddcb54b 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala @@ -24,7 +24,7 @@ import scala.collection.mutable import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.Path -import org.apache.spark.SparkException +import org.apache.spark.{SparkConf, SparkException} import org.apache.spark.sql.AnalysisException import org.apache.spark.sql.catalyst.{FunctionIdentifier, TableIdentifier} import org.apache.spark.sql.catalyst.analysis._ @@ -39,7 +39,11 @@ import org.apache.spark.sql.catalyst.util.StringUtils * * All public methods should be synchronized for thread-safety. */ -class InMemoryCatalog(hadoopConfig: Configuration = new Configuration) extends ExternalCatalog { +class InMemoryCatalog( + conf: SparkConf = new SparkConf, + hadoopConfig: Configuration = new Configuration) + extends ExternalCatalog { + import CatalogTypes.TablePartitionSpec private class TableDesc(var table: CatalogTable) { |