diff options
| author | lianhuiwang <lianhuiwang09@gmail.com> | 2015-01-12 10:57:12 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2015-01-12 10:57:24 -0800 |
| commit | 558be07710cef7a8c2ba1e4237cb6fafdf34981b (patch) | |
| tree | 9bdf1eecb47d6ac2c22dc0e36b696d8ba2eedf1d /core | |
| parent | a1ee09e6777388c201ba5f833bacdf1211c3a08b (diff) | |
| download | spark-558be07710cef7a8c2ba1e4237cb6fafdf34981b.tar.gz spark-558be07710cef7a8c2ba1e4237cb6fafdf34981b.tar.bz2 spark-558be07710cef7a8c2ba1e4237cb6fafdf34981b.zip | |
[SPARK-5102][Core]subclass of MapStatus needs to be registered with Kryo
CompressedMapStatus and HighlyCompressedMapStatus needs to be registered with Kryo, because they are subclass of MapStatus.
Author: lianhuiwang <lianhuiwang09@gmail.com>
Closes #4007 from lianhuiwang/SPARK-5102 and squashes the following commits:
9d2238a [lianhuiwang] remove register of MapStatus
05a285d [lianhuiwang] subclass of MapStatus needs to be registered with Kryo
(cherry picked from commit ef9224e08010420b570c21a0b9208d22792a24fe)
Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala index d2947dcea4..d56e23ce44 100644 --- a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala +++ b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala @@ -29,7 +29,7 @@ import org.apache.spark._ import org.apache.spark.api.python.PythonBroadcast import org.apache.spark.broadcast.HttpBroadcast import org.apache.spark.network.nio.{PutBlock, GotBlock, GetBlock} -import org.apache.spark.scheduler.MapStatus +import org.apache.spark.scheduler.{CompressedMapStatus, HighlyCompressedMapStatus} import org.apache.spark.storage._ import org.apache.spark.util.BoundedPriorityQueue import org.apache.spark.util.collection.CompactBuffer @@ -207,7 +207,8 @@ private[serializer] object KryoSerializer { classOf[PutBlock], classOf[GotBlock], classOf[GetBlock], - classOf[MapStatus], + classOf[CompressedMapStatus], + classOf[HighlyCompressedMapStatus], classOf[CompactBuffer[_]], classOf[BlockManagerId], classOf[Array[Byte]], |