aboutsummaryrefslogtreecommitdiff
path: root/core
diff options
context:
space:
mode:
authorWenchen Fan <cloud0fan@outlook.com>2015-07-25 12:10:02 -0700
committerReynold Xin <rxin@databricks.com>2015-07-25 12:10:02 -0700
commit2c94d0f24a37fa079b56d534b0b0a4574209215b (patch)
tree0aa5ca6ff2c9bb9728bfb6445bd4bfeb4cacf7b7 /core
parente2ec018e37cb699077b5fa2bd662f2055cb42296 (diff)
downloadspark-2c94d0f24a37fa079b56d534b0b0a4574209215b.tar.gz
spark-2c94d0f24a37fa079b56d534b0b0a4574209215b.tar.bz2
spark-2c94d0f24a37fa079b56d534b0b0a4574209215b.zip
[SPARK-9192][SQL] add initialization phase for nondeterministic expression
Currently nondeterministic expression is broken without a explicit initialization phase. Let me take `MonotonicallyIncreasingID` as an example. This expression need a mutable state to remember how many times it has been evaluated, so we use `transient var count: Long` there. By being transient, the `count` will be reset to 0 and **only** to 0 when serialize and deserialize it, as deserialize transient variable will result to default value. There is *no way* to use another initial value for `count`, until we add the explicit initialization phase. Another use case is local execution for `LocalRelation`, there is no serialize and deserialize phase and thus we can't reset mutable states for it. Author: Wenchen Fan <cloud0fan@outlook.com> Closes #7535 from cloud-fan/init and squashes the following commits: 6c6f332 [Wenchen Fan] add test ef68ff4 [Wenchen Fan] fix comments 9eac85e [Wenchen Fan] move init code to interpreted class bb7d838 [Wenchen Fan] pulls out nondeterministic expressions into a project b4a4fc7 [Wenchen Fan] revert a refactor 86fee36 [Wenchen Fan] add initialization phase for nondeterministic expression
Diffstat (limited to 'core')
0 files changed, 0 insertions, 0 deletions