diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-06-04 13:52:53 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-06-04 13:52:53 -0700 |
| commit | 2bcdf8c239d2ba79f64fb8878da83d4c2ec28b30 (patch) | |
| tree | a90fc0d54a9817db37a5d1a61eb7a87b99dbffb0 /project | |
| parent | 65938422718383d17f084e577763e2c671726baa (diff) | |
| download | spark-2bcdf8c239d2ba79f64fb8878da83d4c2ec28b30.tar.gz spark-2bcdf8c239d2ba79f64fb8878da83d4c2ec28b30.tar.bz2 spark-2bcdf8c239d2ba79f64fb8878da83d4c2ec28b30.zip | |
[SPARK-7440][SQL] Remove physical Distinct operator in favor of Aggregate
This patch replaces Distinct with Aggregate in the optimizer, so Distinct will become
more efficient over time as we optimize Aggregate (via Tungsten).
Author: Reynold Xin <rxin@databricks.com>
Closes #6637 from rxin/replace-distinct and squashes the following commits:
b3cc50e [Reynold Xin] Mima excludes.
93d6117 [Reynold Xin] Code review feedback.
87e4741 [Reynold Xin] [SPARK-7440][SQL] Remove physical Distinct operator in favor of Aggregate.
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 34371c9659..73e4bfd78e 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -46,7 +46,9 @@ object MimaExcludes { "org.apache.spark.api.java.JavaRDDLike.partitioner"), // Mima false positive (was a private[spark] class) ProblemFilters.exclude[MissingClassProblem]( - "org.apache.spark.util.collection.PairIterator") + "org.apache.spark.util.collection.PairIterator"), + // SQL execution is considered private. + excludePackage("org.apache.spark.sql.execution") ) case v if v.startsWith("1.4") => Seq( |