diff options
author | Takuya UESHIN <ueshin@happy-camper.st> | 2014-05-26 00:17:20 -0700 |
---|---|---|
committer | Reynold Xin <rxin@apache.org> | 2014-05-26 00:17:20 -0700 |
commit | d6395d86f90d1c47c5b6ad17c618b56e00b7fc85 (patch) | |
tree | ed35a72172a84f362cba1ead3d54bc453a38c3ba /sql/catalyst | |
parent | b6d22af040073cd611b0fcfdf8a5259c0dfd854c (diff) | |
download | spark-d6395d86f90d1c47c5b6ad17c618b56e00b7fc85.tar.gz spark-d6395d86f90d1c47c5b6ad17c618b56e00b7fc85.tar.bz2 spark-d6395d86f90d1c47c5b6ad17c618b56e00b7fc85.zip |
[SPARK-1914] [SQL] Simplify CountFunction not to traverse to evaluate all child expressions.
`CountFunction` should count up only if the child's evaluated value is not null.
Because it traverses to evaluate all child expressions, even if the child is null, it counts up if one of the all children is not null.
Author: Takuya UESHIN <ueshin@happy-camper.st>
Closes #861 from ueshin/issues/SPARK-1914 and squashes the following commits:
3b37315 [Takuya UESHIN] Merge branch 'master' into issues/SPARK-1914
2afa238 [Takuya UESHIN] Simplify CountFunction not to traverse to evaluate all child expressions.
Diffstat (limited to 'sql/catalyst')
-rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala index 1bcd4e2276..79937b129a 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala @@ -298,8 +298,8 @@ case class CountFunction(expr: Expression, base: AggregateExpression) extends Ag var count: Long = _ override def update(input: Row): Unit = { - val evaluatedExpr = expr.map(_.eval(input)) - if (evaluatedExpr.map(_ != null).reduceLeft(_ || _)) { + val evaluatedExpr = expr.eval(input) + if (evaluatedExpr != null) { count += 1L } } |