diff options
author | Josh Rosen <joshrosen@databricks.com> | 2015-07-14 18:55:34 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@databricks.com> | 2015-07-14 18:55:34 -0700 |
commit | cc57d705e732aefc2f3d3f438e84d71705b2eb65 (patch) | |
tree | 12a6b2b2a2cb580bea942765e907a23e0333d96c /streaming | |
parent | e965a798d09a9fba61b104c5cc0b65cdc28d27f6 (diff) | |
download | spark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.tar.gz spark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.tar.bz2 spark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.zip |
[SPARK-9050] [SQL] Remove unused newOrdering argument from Exchange (cleanup after SPARK-8317)
SPARK-8317 changed the SQL Exchange operator so that it no longer pushed sorting into Spark's shuffle layer, a change which allowed more efficient SQL-specific sorters to be used.
This patch performs some leftover cleanup based on those changes:
- Exchange's constructor should no longer accept a `newOrdering` since it's no longer used and no longer works as expected.
- `addOperatorsIfNecessary` looked at shuffle input's output ordering to decide whether to sort, but this is the wrong node to be examining: it needs to look at whether the post-shuffle node has the right ordering, since shuffling will not preserve row orderings. Thanks to davies for spotting this.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #7407 from JoshRosen/SPARK-9050 and squashes the following commits:
e70be50 [Josh Rosen] No need to wrap line
e866494 [Josh Rosen] Refactor addOperatorsIfNecessary to make code clearer
2e467da [Josh Rosen] Remove `newOrdering` from Exchange.
Diffstat (limited to 'streaming')
0 files changed, 0 insertions, 0 deletions