diff options
| author | Davies Liu <davies@databricks.com> | 2015-11-03 11:42:08 +0100 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-11-03 11:42:08 +0100 |
| commit | 67e23b39ac3cdee06668fa9131951278b9731e29 (patch) | |
| tree | eb03786e2392d69c0e6964df7d24739825b35c1c /sql/core/src/test | |
| parent | d728d5c98658c44ed2949b55d36edeaa46f8c980 (diff) | |
| download | spark-67e23b39ac3cdee06668fa9131951278b9731e29.tar.gz spark-67e23b39ac3cdee06668fa9131951278b9731e29.tar.bz2 spark-67e23b39ac3cdee06668fa9131951278b9731e29.zip | |
[SPARK-10429] [SQL] make mutableProjection atomic
Right now, SQL's mutable projection updates every value of the mutable project after it evaluates the corresponding expression. This makes the behavior of MutableProjection confusing and complicate the implementation of common aggregate functions like stddev because developers need to be aware that when evaluating {{i+1}}th expression of a mutable projection, {{i}}th slot of the mutable row has already been updated.
This PR make the MutableProjection atomic, by generating all the results of expressions first, then copy them into mutableRow.
Had run a mircro-benchmark, there is no notable performance difference between using class members and local variables.
cc yhuai
Author: Davies Liu <davies@databricks.com>
Closes #9422 from davies/atomic_mutable and squashes the following commits:
bbc1758 [Davies Liu] support wide table
8a0ae14 [Davies Liu] fix bug
bec07da [Davies Liu] refactor
2891628 [Davies Liu] make mutableProjection atomic
Diffstat (limited to 'sql/core/src/test')
0 files changed, 0 insertions, 0 deletions