diff options
author | Reynold Xin <rxin@databricks.com> | 2016-07-12 10:07:23 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-07-12 10:07:23 -0700 |
commit | c377e49e38a290e5c4fbc178278069788674dfb7 (patch) | |
tree | ef043d59d8ab9eb0b778fe6a703cf73bfee6cfaf /sql/catalyst/src/main/scala | |
parent | 5ad68ba5ce625c7005b540ca50ed001ca18de967 (diff) | |
download | spark-c377e49e38a290e5c4fbc178278069788674dfb7.tar.gz spark-c377e49e38a290e5c4fbc178278069788674dfb7.tar.bz2 spark-c377e49e38a290e5c4fbc178278069788674dfb7.zip |
[SPARK-16489][SQL] Guard against variable reuse mistakes in expression code generation
## What changes were proposed in this pull request?
In code generation, it is incorrect for expressions to reuse variable names across different instances of itself. As an example, SPARK-16488 reports a bug in which pmod expression reuses variable name "r".
This patch updates ExpressionEvalHelper test harness to always project two instances of the same expression, which will help us catch variable reuse problems in expression unit tests. This patch also fixes the bug in crc32 expression.
## How was this patch tested?
This is a test harness change, but I also created a new test suite for testing the test harness.
Author: Reynold Xin <rxin@databricks.com>
Closes #14146 from rxin/SPARK-16489.
Diffstat (limited to 'sql/catalyst/src/main/scala')
-rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala | 7 |
1 files changed, 4 insertions, 3 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala index 1c0787bf92..d2c94ec1df 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala @@ -175,11 +175,12 @@ case class Crc32(child: Expression) extends UnaryExpression with ImplicitCastInp override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = { val CRC32 = "java.util.zip.CRC32" + val checksum = ctx.freshName("checksum") nullSafeCodeGen(ctx, ev, value => { s""" - $CRC32 checksum = new $CRC32(); - checksum.update($value, 0, $value.length); - ${ev.value} = checksum.getValue(); + $CRC32 $checksum = new $CRC32(); + $checksum.update($value, 0, $value.length); + ${ev.value} = $checksum.getValue(); """ }) } |