diff options
| author | Cheng Lian <lian@databricks.com> | 2016-08-23 09:11:47 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-08-23 09:11:47 +0800 |
| commit | 2cdd92a7cd6f85186c846635b422b977bdafbcdd (patch) | |
| tree | fd6dffddf896ba130a9bcd6a82c6dbd780dd8199 /sql | |
| parent | 920806ab272ba58a369072a5eeb89df5e9b470a6 (diff) | |
| download | spark-2cdd92a7cd6f85186c846635b422b977bdafbcdd.tar.gz spark-2cdd92a7cd6f85186c846635b422b977bdafbcdd.tar.bz2 spark-2cdd92a7cd6f85186c846635b422b977bdafbcdd.zip | |
[SPARK-17182][SQL] Mark Collect as non-deterministic
## What changes were proposed in this pull request?
This PR marks the abstract class `Collect` as non-deterministic since the results of `CollectList` and `CollectSet` depend on the actual order of input rows.
## How was this patch tested?
Existing test cases should be enough.
Author: Cheng Lian <lian@databricks.com>
Closes #14749 from liancheng/spark-17182-non-deterministic-collect.
Diffstat (limited to 'sql')
| -rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala index ac2cefaddc..896ff61b23 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/collect.scala @@ -54,6 +54,10 @@ abstract class Collect extends ImperativeAggregate { override def inputAggBufferAttributes: Seq[AttributeReference] = Nil + // Both `CollectList` and `CollectSet` are non-deterministic since their results depend on the + // actual order of input rows. + override def deterministic: Boolean = false + protected[this] val buffer: Growable[Any] with Iterable[Any] override def initialize(b: MutableRow): Unit = { |