diff options
| author | Michael Armbrust <michael@databricks.com> | 2014-07-04 19:15:48 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-07-04 19:16:08 -0700 |
| commit | b77715a5bd46ab31b8dbcaa36555c9f3312b72c4 (patch) | |
| tree | fe8146be7013f8df7ce9869f5d91ab7a19270f42 /sql/hive | |
| parent | d9b5a8e2fd1dd0c02ea630ebfb91728fe463e1d6 (diff) | |
| download | spark-b77715a5bd46ab31b8dbcaa36555c9f3312b72c4.tar.gz spark-b77715a5bd46ab31b8dbcaa36555c9f3312b72c4.tar.bz2 spark-b77715a5bd46ab31b8dbcaa36555c9f3312b72c4.zip | |
[SPARK-2370][SQL] Decrease metadata retrieved for partitioned hive queries.
Author: Michael Armbrust <michael@databricks.com>
Closes #1305 from marmbrus/usePrunerPartitions and squashes the following commits:
744aa20 [Michael Armbrust] Use getAllPartitionsForPruner instead of getPartitions, which avoids retrieving auth data
(cherry picked from commit 9d006c97371ddf357e0b821d5c6d1535d9b6fe41)
Signed-off-by: Reynold Xin <rxin@apache.org>
Diffstat (limited to 'sql/hive')
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala index faa30c9ae5..b3dba14519 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala @@ -54,7 +54,7 @@ private[hive] class HiveMetastoreCatalog(hive: HiveContext) extends Catalog with val table = client.getTable(databaseName, tableName) val partitions: Seq[Partition] = if (table.isPartitioned) { - client.getPartitions(table) + client.getAllPartitionsForPruner(table).toSeq } else { Nil } |