aboutsummaryrefslogtreecommitdiff
path: root/streaming/src/main
diff options
context:
space:
mode:
authorJosh Rosen <joshrosen@databricks.com>2015-07-14 18:55:34 -0700
committerJosh Rosen <joshrosen@databricks.com>2015-07-14 18:55:34 -0700
commitcc57d705e732aefc2f3d3f438e84d71705b2eb65 (patch)
tree12a6b2b2a2cb580bea942765e907a23e0333d96c /streaming/src/main
parente965a798d09a9fba61b104c5cc0b65cdc28d27f6 (diff)
downloadspark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.tar.gz
spark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.tar.bz2
spark-cc57d705e732aefc2f3d3f438e84d71705b2eb65.zip
[SPARK-9050] [SQL] Remove unused newOrdering argument from Exchange (cleanup after SPARK-8317)
SPARK-8317 changed the SQL Exchange operator so that it no longer pushed sorting into Spark's shuffle layer, a change which allowed more efficient SQL-specific sorters to be used. This patch performs some leftover cleanup based on those changes: - Exchange's constructor should no longer accept a `newOrdering` since it's no longer used and no longer works as expected. - `addOperatorsIfNecessary` looked at shuffle input's output ordering to decide whether to sort, but this is the wrong node to be examining: it needs to look at whether the post-shuffle node has the right ordering, since shuffling will not preserve row orderings. Thanks to davies for spotting this. Author: Josh Rosen <joshrosen@databricks.com> Closes #7407 from JoshRosen/SPARK-9050 and squashes the following commits: e70be50 [Josh Rosen] No need to wrap line e866494 [Josh Rosen] Refactor addOperatorsIfNecessary to make code clearer 2e467da [Josh Rosen] Remove `newOrdering` from Exchange.
Diffstat (limited to 'streaming/src/main')
0 files changed, 0 insertions, 0 deletions