diff options
| author | Daoyuan Wang <daoyuan.wang@intel.com> | 2015-04-13 16:00:58 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-04-13 16:00:58 -0700 |
| commit | e63a86abe2794332cdad71d87b72a7c56327a43d (patch) | |
| tree | 2040d929b119e2fb33caf7e7972daaef3f78ab38 /sql/core | |
| parent | 2a55cb41bf7da1786be2c76b8af398da8fedb44b (diff) | |
| download | spark-e63a86abe2794332cdad71d87b72a7c56327a43d.tar.gz spark-e63a86abe2794332cdad71d87b72a7c56327a43d.tar.bz2 spark-e63a86abe2794332cdad71d87b72a7c56327a43d.zip | |
[SPARK-6872] [SQL] add copy in external sort
We need add copy before call externalsort.
Author: Daoyuan Wang <daoyuan.wang@intel.com>
Closes #5481 from adrian-wang/extsort and squashes the following commits:
9611586 [Daoyuan Wang] fix bug in external sort
Diffstat (limited to 'sql/core')
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala index 6eec520abf..f8221f41bc 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala @@ -194,7 +194,7 @@ case class ExternalSort( child.execute().mapPartitions( { iterator => val ordering = newOrdering(sortOrder, child.output) val sorter = new ExternalSorter[Row, Null, Row](ordering = Some(ordering)) - sorter.insertAll(iterator.map(r => (r, null))) + sorter.insertAll(iterator.map(r => (r.copy, null))) val baseIterator = sorter.iterator.map(_._1) // TODO(marmbrus): The complex type signature below thwarts inference for no reason. CompletionIterator[Row, Iterator[Row]](baseIterator, sorter.stop()) |