aboutsummaryrefslogtreecommitdiff
path: root/dev/scalastyle
diff options
context:
space:
mode:
authorWenchen Fan <wenchen@databricks.com>2016-02-09 13:06:36 -0800
committerDavies Liu <davies.liu@gmail.com>2016-02-09 13:06:36 -0800
commit7fe4fe630a3fc9755ebd0325bb595d76381633e8 (patch)
treecbeebfe9ba88887e8e29f7e4b4aeaf73cbab4f8b /dev/scalastyle
parent2dbb9164405d6f595905c7d4b32e20177f0f669f (diff)
downloadspark-7fe4fe630a3fc9755ebd0325bb595d76381633e8.tar.gz
spark-7fe4fe630a3fc9755ebd0325bb595d76381633e8.tar.bz2
spark-7fe4fe630a3fc9755ebd0325bb595d76381633e8.zip
[SPARK-12888] [SQL] [FOLLOW-UP] benchmark the new hash expression
Adds the benchmark results as comments. The codegen version is slower than the interpreted version for `simple` case becasue of 3 reasons: 1. codegen version use a more complex hash algorithm than interpreted version, i.e. `Murmur3_x86_32.hashInt` vs [simple multiplication and addition](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/rows.scala#L153). 2. codegen version will write the hash value to a row first and then read it out. I tried to create a `GenerateHasher` that can generate code to return hash value directly and got about 60% speed up for the `simple` case, does it worth? 3. the row in `simple` case only has one int field, so the runtime reflection may be removed because of branch prediction, which makes the interpreted version faster. The `array` case is also slow for similar reasons, e.g. array elements are of same type, so interpreted version can probably get rid of runtime reflection by branch prediction. Author: Wenchen Fan <wenchen@databricks.com> Closes #10917 from cloud-fan/hash-benchmark.
Diffstat (limited to 'dev/scalastyle')
0 files changed, 0 insertions, 0 deletions