aboutsummaryrefslogtreecommitdiff
path: root/pom.xml
diff options
context:
space:
mode:
authorCheng Hao <hao.cheng@intel.com>2014-05-15 22:12:34 -0700
committerReynold Xin <rxin@apache.org>2014-05-15 22:12:34 -0700
commita20fea98811d98958567780815fcf0d4fb4e28d4 (patch)
treee9c3ea380653ca6168016f079309d5caefdd17da /pom.xml
parentbb98ecafce196ecc5bc3a1e4cc9264df7b752c6a (diff)
downloadspark-a20fea98811d98958567780815fcf0d4fb4e28d4.tar.gz
spark-a20fea98811d98958567780815fcf0d4fb4e28d4.tar.bz2
spark-a20fea98811d98958567780815fcf0d4fb4e28d4.zip
[Spark-1461] Deferred Expression Evaluation (short-circuit evaluation)
This patch unify the foldable & nullable interface for Expression. 1) Deterministic-less UDF (like Rand()) can not be folded. 2) Short-circut will significantly improves the performance in Expression Evaluation, however, the stateful UDF should not be ignored in a short-circuit evaluation(e.g. in expression: col1 > 0 and row_sequence() < 1000, row_sequence() can not be ignored even if col1 > 0 is false) I brought an concept of DeferredObject from Hive, which has 2 kinds of children classes (EagerResult / DeferredResult), the former requires triggering the evaluation before it's created, while the later trigger the evaluation when first called its get() method. Author: Cheng Hao <hao.cheng@intel.com> Closes #446 from chenghao-intel/expression_deferred_evaluation and squashes the following commits: d2729de [Cheng Hao] Fix the codestyle issues a08f09c [Cheng Hao] fix bug in or/and short-circuit evaluation af2236b [Cheng Hao] revert the short-circuit expression evaluation for IF b7861d2 [Cheng Hao] Add Support for Deferred Expression Evaluation
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions