diff options
author | Reynold Xin <rxin@databricks.com> | 2015-07-29 20:46:03 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2015-07-29 20:46:03 -0700 |
commit | 9514d874f0cf61f1eb4ec4f5f66e053119f769c9 (patch) | |
tree | ed692e94ed59d559b4b2b16ca8b88045021bbd6b /sql/catalyst | |
parent | a200e64561c8803731578267df16906f6773cbea (diff) | |
download | spark-9514d874f0cf61f1eb4ec4f5f66e053119f769c9.tar.gz spark-9514d874f0cf61f1eb4ec4f5f66e053119f769c9.tar.bz2 spark-9514d874f0cf61f1eb4ec4f5f66e053119f769c9.zip |
[SPARK-9458] Avoid object allocation in prefix generation.
In our existing sort prefix generation code, we use expression's eval method to generate the prefix, which results in object allocation for every prefix. We can use the specialized getters available on InternalRow directly to avoid the object allocation.
I also removed the FLOAT prefix, opting for converting float directly to double.
Author: Reynold Xin <rxin@databricks.com>
Closes #7763 from rxin/sort-prefix and squashes the following commits:
5dc2f06 [Reynold Xin] [SPARK-9458] Avoid object allocation in prefix generation.
Diffstat (limited to 'sql/catalyst')
-rw-r--r-- | sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java b/sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java index 4c3f2c6557..8342833246 100644 --- a/sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java +++ b/sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java @@ -121,7 +121,7 @@ final class UnsafeExternalRowSorter { // here in order to prevent memory leaks. cleanupResources(); } - return new AbstractScalaRowIterator() { + return new AbstractScalaRowIterator<InternalRow>() { private final int numFields = schema.length(); private UnsafeRow row = new UnsafeRow(); |