diff options
| author | Reynold Xin <rxin@databricks.com> | 2017-03-16 18:31:57 -0700 |
|---|---|---|
| committer | Xiao Li <gatorsmile@gmail.com> | 2017-03-16 18:31:57 -0700 |
| commit | 8537c00e0a17eff2a8c6745fbdd1d08873c0434d (patch) | |
| tree | 79e01f299fa6d76bd87f79315e2a18a86a685880 /sql/hive | |
| parent | 4c3200546c5c55e671988a957011417ba76a0600 (diff) | |
| download | spark-8537c00e0a17eff2a8c6745fbdd1d08873c0434d.tar.gz spark-8537c00e0a17eff2a8c6745fbdd1d08873c0434d.tar.bz2 spark-8537c00e0a17eff2a8c6745fbdd1d08873c0434d.zip | |
[SPARK-19987][SQL] Pass all filters into FileIndex
## What changes were proposed in this pull request?
This is a tiny teeny refactoring to pass data filters also to the FileIndex, so FileIndex can have a more global view on predicates.
## How was this patch tested?
Change should be covered by existing test cases.
Author: Reynold Xin <rxin@databricks.com>
Closes #17322 from rxin/SPARK-19987.
Diffstat (limited to 'sql/hive')
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala index 9f0d1ceb28..2e060ab9f6 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala @@ -17,8 +17,6 @@ package org.apache.spark.sql.hive -import java.net.URI - import scala.util.control.NonFatal import com.google.common.util.concurrent.Striped @@ -248,7 +246,7 @@ private[hive] class HiveMetastoreCatalog(sparkSession: SparkSession) extends Log .inferSchema( sparkSession, options, - fileIndex.listFiles(Nil).flatMap(_.files)) + fileIndex.listFiles(Nil, Nil).flatMap(_.files)) .map(mergeWithMetastoreSchema(relation.tableMeta.schema, _)) inferredSchema match { |