diff options
| author | jerryshao <saisai.shao@intel.com> | 2014-06-23 20:25:46 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@databricks.com> | 2014-06-23 20:25:46 -0700 |
| commit | 56eb8af187b19f09810baafb314b21e07cf0a79c (patch) | |
| tree | 30bcca83114d3c28659876c6da593eda2241f30d /docs | |
| parent | 51c8168377a89d20d0b2d7b9a28af58593a0fe0c (diff) | |
| download | spark-56eb8af187b19f09810baafb314b21e07cf0a79c.tar.gz spark-56eb8af187b19f09810baafb314b21e07cf0a79c.tar.bz2 spark-56eb8af187b19f09810baafb314b21e07cf0a79c.zip | |
[SPARK-2124] Move aggregation into shuffle implementations
This PR is a sub-task of SPARK-2044 to move the execution of aggregation into shuffle implementations.
I leave `CoGoupedRDD` and `SubtractedRDD` unchanged because they have their implementations of aggregation. I'm not sure is it suitable to change these two RDDs.
Also I do not move sort related code of `OrderedRDDFunctions` into shuffle, this will be solved in another sub-task.
Author: jerryshao <saisai.shao@intel.com>
Closes #1064 from jerryshao/SPARK-2124 and squashes the following commits:
4a05a40 [jerryshao] Modify according to comments
1f7dcc8 [jerryshao] Style changes
50a2fd6 [jerryshao] Fix test suite issue after moving aggregator to Shuffle reader and writer
1a96190 [jerryshao] Code modification related to the ShuffledRDD
308f635 [jerryshao] initial works of move combiner to ShuffleManager's reader and writer
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions