diff options
author | Josh Rosen <joshrosen@databricks.com> | 2015-10-22 09:46:30 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@databricks.com> | 2015-10-22 09:46:30 -0700 |
commit | f6d06adf05afa9c5386dc2396c94e7a98730289f (patch) | |
tree | 9e3d8e4350e0a465124840eea91f6aa39c00b156 /project/MimaExcludes.scala | |
parent | 94e2064fa1b04c05c805d9175c7c78bf583db5c6 (diff) | |
download | spark-f6d06adf05afa9c5386dc2396c94e7a98730289f.tar.gz spark-f6d06adf05afa9c5386dc2396c94e7a98730289f.tar.bz2 spark-f6d06adf05afa9c5386dc2396c94e7a98730289f.zip |
[SPARK-10708] Consolidate sort shuffle implementations
There's a lot of duplication between SortShuffleManager and UnsafeShuffleManager. Given that these now provide the same set of functionality, now that UnsafeShuffleManager supports large records, I think that we should replace SortShuffleManager's serialized shuffle implementation with UnsafeShuffleManager's and should merge the two managers together.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #8829 from JoshRosen/consolidate-sort-shuffle-implementations.
Diffstat (limited to 'project/MimaExcludes.scala')
-rw-r--r-- | project/MimaExcludes.scala | 9 |
1 files changed, 7 insertions, 2 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 0872d3f3e7..b5e661d3ec 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -37,6 +37,7 @@ object MimaExcludes { Seq( MimaBuild.excludeSparkPackage("deploy"), MimaBuild.excludeSparkPackage("network"), + MimaBuild.excludeSparkPackage("unsafe"), // These are needed if checking against the sbt build, since they are part of // the maven-generated artifacts in 1.3. excludePackage("org.spark-project.jetty"), @@ -44,7 +45,11 @@ object MimaExcludes { // SQL execution is considered private. excludePackage("org.apache.spark.sql.execution"), // SQL columnar is considered private. - excludePackage("org.apache.spark.sql.columnar") + excludePackage("org.apache.spark.sql.columnar"), + // The shuffle package is considered private. + excludePackage("org.apache.spark.shuffle"), + // The collections utlities are considered pricate. + excludePackage("org.apache.spark.util.collection") ) ++ MimaBuild.excludeSparkClass("streaming.flume.FlumeTestUtils") ++ MimaBuild.excludeSparkClass("streaming.flume.PollingFlumeTestUtils") ++ @@ -750,4 +755,4 @@ object MimaExcludes { MimaBuild.excludeSparkClass("mllib.regression.LinearRegressionWithSGD") case _ => Seq() } -}
\ No newline at end of file +} |