diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-08-01 13:20:26 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-08-01 13:20:26 -0700 |
| commit | 3d1535d48822281953de1e8447de86fad728412a (patch) | |
| tree | 52f12ad351ecc49c7e2563e0c19e6fa4c622476b /unsafe | |
| parent | df733cbeae7a53826e89574af5463fa018329a22 (diff) | |
| download | spark-3d1535d48822281953de1e8447de86fad728412a.tar.gz spark-3d1535d48822281953de1e8447de86fad728412a.tar.bz2 spark-3d1535d48822281953de1e8447de86fad728412a.zip | |
[SPARK-9520] [SQL] Support in-place sort in UnsafeFixedWidthAggregationMap
This pull request adds a sortedIterator method to UnsafeFixedWidthAggregationMap that sorts its data in-place by the grouping key.
This is needed so we can fallback to external sorting for aggregation.
Author: Reynold Xin <rxin@databricks.com>
Closes #7849 from rxin/bytes2bytes-sorting and squashes the following commits:
75018c6 [Reynold Xin] Updated documentation.
81a8694 [Reynold Xin] [SPARK-9520][SQL] Support in-place sort in UnsafeFixedWidthAggregationMap.
Diffstat (limited to 'unsafe')
| -rw-r--r-- | unsafe/src/main/java/org/apache/spark/unsafe/KVIterator.java | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/unsafe/src/main/java/org/apache/spark/unsafe/KVIterator.java b/unsafe/src/main/java/org/apache/spark/unsafe/KVIterator.java index fb163401c0..5c9d5d9a38 100644 --- a/unsafe/src/main/java/org/apache/spark/unsafe/KVIterator.java +++ b/unsafe/src/main/java/org/apache/spark/unsafe/KVIterator.java @@ -17,9 +17,11 @@ package org.apache.spark.unsafe; +import java.io.IOException; + public abstract class KVIterator<K, V> { - public abstract boolean next(); + public abstract boolean next() throws IOException; public abstract K getKey(); |