diff options
author | Sean Zhong <seanzhong@databricks.com> | 2016-09-21 16:53:34 +0800 |
---|---|---|
committer | Wenchen Fan <wenchen@databricks.com> | 2016-09-21 16:53:34 +0800 |
commit | 3977223a3268aaf6913a325ee459139a4a302b1c (patch) | |
tree | ad0b596e5bdc22821abd2e215712eb672ae54037 /sql/catalyst/src/main/scala/org/apache | |
parent | 7654385f268a3f481c4574ce47a19ab21155efd5 (diff) | |
download | spark-3977223a3268aaf6913a325ee459139a4a302b1c.tar.gz spark-3977223a3268aaf6913a325ee459139a4a302b1c.tar.bz2 spark-3977223a3268aaf6913a325ee459139a4a302b1c.zip |
[SPARK-17617][SQL] Remainder(%) expression.eval returns incorrect result on double value
## What changes were proposed in this pull request?
Remainder(%) expression's `eval()` returns incorrect result when the dividend is a big double. The reason is that Remainder converts the double dividend to decimal to do "%", and that lose precision.
This bug only affects the `eval()` that is used by constant folding, the codegen path is not impacted.
### Before change
```
scala> -5083676433652386516D % 10
res2: Double = -6.0
scala> spark.sql("select -5083676433652386516D % 10 as a").show
+---+
| a|
+---+
|0.0|
+---+
```
### After change
```
scala> spark.sql("select -5083676433652386516D % 10 as a").show
+----+
| a|
+----+
|-6.0|
+----+
```
## How was this patch tested?
Unit test.
Author: Sean Zhong <seanzhong@databricks.com>
Closes #15171 from clockfly/SPARK-17617.
Diffstat (limited to 'sql/catalyst/src/main/scala/org/apache')
-rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala index 13e539a223..6f3db79622 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala @@ -310,7 +310,11 @@ case class Remainder(left: Expression, right: Expression) if (input1 == null) { null } else { - integral.rem(input1, input2) + input1 match { + case d: Double => d % input2.asInstanceOf[java.lang.Double] + case f: Float => f % input2.asInstanceOf[java.lang.Float] + case _ => integral.rem(input1, input2) + } } } } |