diff options
| author | Daoyuan Wang <daoyuan.wang@intel.com> | 2014-12-11 22:49:27 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-12-11 22:49:27 -0800 |
| commit | acb3be6bc5e0e793f769ec371dffeb474dc0cf21 (patch) | |
| tree | 0bdc9466042ea3cda26e41b34a42335ccd84c80c /sql/catalyst | |
| parent | d8cf67858988bed287c0125095729f7f808aabad (diff) | |
| download | spark-acb3be6bc5e0e793f769ec371dffeb474dc0cf21.tar.gz spark-acb3be6bc5e0e793f769ec371dffeb474dc0cf21.tar.bz2 spark-acb3be6bc5e0e793f769ec371dffeb474dc0cf21.zip | |
[SPARK-4828] [SQL] sum and avg on empty table should always return null
So the optimizations are not valid. Also I think the optimization here is rarely encounter, so removing them will not have influence on performance.
Can we merge #3445 before I add a comparison test case from this?
Author: Daoyuan Wang <daoyuan.wang@intel.com>
Closes #3675 from adrian-wang/sumempty and squashes the following commits:
42df763 [Daoyuan Wang] sum and avg on empty table should always return null
Diffstat (limited to 'sql/catalyst')
| -rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala | 2 |
1 files changed, 0 insertions, 2 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala index f164a6c68a..c492348959 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala @@ -203,8 +203,6 @@ object NullPropagation extends Rule[LogicalPlan] { def apply(plan: LogicalPlan): LogicalPlan = plan transform { case q: LogicalPlan => q transformExpressionsUp { case e @ Count(Literal(null, _)) => Cast(Literal(0L), e.dataType) - case e @ Sum(Literal(c, _)) if c == 0 => Cast(Literal(0L), e.dataType) - case e @ Average(Literal(c, _)) if c == 0 => Literal(0.0, e.dataType) case e @ IsNull(c) if !c.nullable => Literal(false, BooleanType) case e @ IsNotNull(c) if !c.nullable => Literal(true, BooleanType) case e @ GetItem(Literal(null, _), _) => Literal(null, e.dataType) |