diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-10-25 08:42:21 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-10-25 08:42:21 +0800 |
| commit | 84a33999082af88ea6365cdb5c7232ed0933b1c6 (patch) | |
| tree | a250c838432818f6ca3ec90b1a8330cbd8c0bc93 /.gitattributes | |
| parent | 407c3cedf29a4413339dcde758295dc3225a0054 (diff) | |
| download | spark-84a33999082af88ea6365cdb5c7232ed0933b1c6.tar.gz spark-84a33999082af88ea6365cdb5c7232ed0933b1c6.tar.bz2 spark-84a33999082af88ea6365cdb5c7232ed0933b1c6.zip | |
[SPARK-18028][SQL] simplify TableFileCatalog
## What changes were proposed in this pull request?
Simplify/cleanup TableFileCatalog:
1. pass a `CatalogTable` instead of `databaseName` and `tableName` into `TableFileCatalog`, so that we don't need to fetch table metadata from metastore again
2. In `TableFileCatalog.filterPartitions0`, DO NOT set `PartitioningAwareFileCatalog.BASE_PATH_PARAM`. According to the [classdoc](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala#L189-L209), the default value of `basePath` already satisfies our need. What's more, if we set this parameter, we may break the case 2 which is metioned in the classdoc.
3. add `equals` and `hashCode` to `TableFileCatalog`
4. add `SessionCatalog.listPartitionsByFilter` which handles case sensitivity.
## How was this patch tested?
existing tests.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #15568 from cloud-fan/table-file-catalog.
Diffstat (limited to '.gitattributes')
0 files changed, 0 insertions, 0 deletions