diff options
author | Takuya UESHIN <ueshin@happy-camper.st> | 2014-05-27 14:55:23 -0700 |
---|---|---|
committer | Reynold Xin <rxin@apache.org> | 2014-05-27 14:55:23 -0700 |
commit | 3b0babad1f0856ee16f9d58e1ead30779a4a6310 (patch) | |
tree | 9f191295ddce1a4554b7a12c201718e4c2703e70 /sql/core | |
parent | d1375a2bff846f2c4274e14545924646852895f9 (diff) | |
download | spark-3b0babad1f0856ee16f9d58e1ead30779a4a6310.tar.gz spark-3b0babad1f0856ee16f9d58e1ead30779a4a6310.tar.bz2 spark-3b0babad1f0856ee16f9d58e1ead30779a4a6310.zip |
[SPARK-1915] [SQL] AverageFunction should not count if the evaluated value is null.
Average values are difference between the calculation is done partially or not partially.
Because `AverageFunction` (in not-partially calculation) counts even if the evaluated value is null.
Author: Takuya UESHIN <ueshin@happy-camper.st>
Closes #862 from ueshin/issues/SPARK-1915 and squashes the following commits:
b1ff3c0 [Takuya UESHIN] Modify AverageFunction not to count if the evaluated value is null.
Diffstat (limited to 'sql/core')
-rw-r--r-- | sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala index 8197e8a18d..fb599e1e01 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala @@ -115,6 +115,16 @@ class DslQuerySuite extends QueryTest { 2.0) } + test("null average") { + checkAnswer( + testData3.groupBy()(Average('b)), + 2.0) + + checkAnswer( + testData3.groupBy()(Average('b), CountDistinct('b :: Nil)), + (2.0, 1) :: Nil) + } + test("count") { assert(testData2.count() === testData2.map(_ => 1).count()) } |