diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-07-20 18:37:15 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2016-07-20 18:37:15 -0700 |
| commit | cfa5ae84ed0f48b3b108d0614dbf6fcd79ef5179 (patch) | |
| tree | 1b529627a22b96cfc04cea4506c717a3737f1698 /sql/catalyst/src/main | |
| parent | 75a06aa256aa256c112555609a93c1e1dbb1cb4b (diff) | |
| download | spark-cfa5ae84ed0f48b3b108d0614dbf6fcd79ef5179.tar.gz spark-cfa5ae84ed0f48b3b108d0614dbf6fcd79ef5179.tar.bz2 spark-cfa5ae84ed0f48b3b108d0614dbf6fcd79ef5179.zip | |
[SPARK-16644][SQL] Aggregate should not propagate constraints containing aggregate expressions
## What changes were proposed in this pull request?
aggregate expressions can only be executed inside `Aggregate`, if we propagate it up with constraints, the parent operator can not execute it and will fail at runtime.
## How was this patch tested?
new test in SQLQuerySuite
Author: Wenchen Fan <wenchen@databricks.com>
Author: Yin Huai <yhuai@databricks.com>
Closes #14281 from cloud-fan/bug.
Diffstat (limited to 'sql/catalyst/src/main')
| -rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala index c0e400f617..b31f5aa11c 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala @@ -483,8 +483,10 @@ case class Aggregate( override def output: Seq[Attribute] = aggregateExpressions.map(_.toAttribute) override def maxRows: Option[Long] = child.maxRows - override def validConstraints: Set[Expression] = - child.constraints.union(getAliasedConstraints(aggregateExpressions)) + override def validConstraints: Set[Expression] = { + val nonAgg = aggregateExpressions.filter(_.find(_.isInstanceOf[AggregateExpression]).isEmpty) + child.constraints.union(getAliasedConstraints(nonAgg)) + } override lazy val statistics: Statistics = { if (groupingExpressions.isEmpty) { |