diff options
author | Kay Ousterhout <kayousterhout@gmail.com> | 2015-07-21 01:12:51 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@databricks.com> | 2015-07-21 01:12:51 -0700 |
commit | 6364735bcc67ecb0e9c7e5076d214ed88e927430 (patch) | |
tree | 24390d99e8fa212707ede920f0ccf8ee12fcff2b /data/mllib/pagerank_data.txt | |
parent | ae230596b866d8e369bd061256c4cc569dba430a (diff) | |
download | spark-6364735bcc67ecb0e9c7e5076d214ed88e927430.tar.gz spark-6364735bcc67ecb0e9c7e5076d214ed88e927430.tar.bz2 spark-6364735bcc67ecb0e9c7e5076d214ed88e927430.zip |
[SPARK-8875] Remove BlockStoreShuffleFetcher class
The shuffle code has gotten increasingly difficult to read as it has evolved, and many classes
have evolved significantly since they were originally created. The BlockStoreShuffleFetcher class
now serves little purpose other than to make the code more difficult to read; this commit moves its
functionality into the ShuffleBlockFetcherIterator class.
cc massie JoshRosen (Josh, this PR also removes the Try you pointed out as being confusing / not necessarily useful in a previous comment). Matt, would be helpful to know whether this will interfere in any negative ways with your new shuffle PR (I took a look and it seems like this should still cleanly integrate with your parquet work, but want to double check).
Author: Kay Ousterhout <kayousterhout@gmail.com>
Closes #7268 from kayousterhout/SPARK-8875 and squashes the following commits:
2b24a97 [Kay Ousterhout] Fixed DAGSchedulerSuite compile error
98a1831 [Kay Ousterhout] Merge remote-tracking branch 'upstream/master' into SPARK-8875
90f0e89 [Kay Ousterhout] Fixed broken test
14bfcbb [Kay Ousterhout] Last style fix
bc69d2b [Kay Ousterhout] Style improvements based on Josh's code review
ad3c8d1 [Kay Ousterhout] Better documentation for MapOutputTracker methods
0bc0e59 [Kay Ousterhout] [SPARK-8875] Remove BlockStoreShuffleFetcher class
Diffstat (limited to 'data/mllib/pagerank_data.txt')
0 files changed, 0 insertions, 0 deletions