diff options
| author | petermaxlee <petermaxlee@gmail.com> | 2016-07-11 22:23:32 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-07-11 22:23:32 -0700 |
| commit | c9a6762150cfd62691a6361e05d2839b110fe8d0 (patch) | |
| tree | 628a1d6e9c6f4f1c984d05bfab435e049d549ebf /project | |
| parent | b1e5281c5cb429e338c3719c13c0b93078d7312a (diff) | |
| download | spark-c9a6762150cfd62691a6361e05d2839b110fe8d0.tar.gz spark-c9a6762150cfd62691a6361e05d2839b110fe8d0.tar.bz2 spark-c9a6762150cfd62691a6361e05d2839b110fe8d0.zip | |
[SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter
## What changes were proposed in this pull request?
It would be useful to support listing the columns that are referenced by a filter. This can help simplify data source planning, because with this we would be able to implement unhandledFilters method in HadoopFsRelation.
This is based on rxin's patch (#13901) and adds unit tests.
## How was this patch tested?
Added a new suite FiltersSuite.
Author: petermaxlee <petermaxlee@gmail.com>
Author: Reynold Xin <rxin@databricks.com>
Closes #14120 from petermaxlee/SPARK-16199.
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 4bd6156288..56061559fe 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -35,7 +35,12 @@ import com.typesafe.tools.mima.core.ProblemFilters._ object MimaExcludes { // Exclude rules for 2.1.x - lazy val v21excludes = v20excludes + lazy val v21excludes = v20excludes ++ { + Seq( + // [SPARK-16199][SQL] Add a method to list the referenced columns in data source Filter + ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.sql.sources.Filter.references") + ) + } // Exclude rules for 2.0.x lazy val v20excludes = { |