diff options
| author | lianhuiwang <lianhuiwang09@gmail.com> | 2015-01-12 10:57:12 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2015-01-12 10:57:12 -0800 |
| commit | ef9224e08010420b570c21a0b9208d22792a24fe (patch) | |
| tree | 26ffd9a5d7ee0f2f60daa3fe7f5435b18564a47f /core | |
| parent | 82fd38dcdcc9f7df18930c0e08cc8ec34eaee828 (diff) | |
| download | spark-ef9224e08010420b570c21a0b9208d22792a24fe.tar.gz spark-ef9224e08010420b570c21a0b9208d22792a24fe.tar.bz2 spark-ef9224e08010420b570c21a0b9208d22792a24fe.zip | |
[SPARK-5102][Core]subclass of MapStatus needs to be registered with Kryo
CompressedMapStatus and HighlyCompressedMapStatus needs to be registered with Kryo, because they are subclass of MapStatus.
Author: lianhuiwang <lianhuiwang09@gmail.com>
Closes #4007 from lianhuiwang/SPARK-5102 and squashes the following commits:
9d2238a [lianhuiwang] remove register of MapStatus
05a285d [lianhuiwang] subclass of MapStatus needs to be registered with Kryo
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala index d2947dcea4..d56e23ce44 100644 --- a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala +++ b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala @@ -29,7 +29,7 @@ import org.apache.spark._ import org.apache.spark.api.python.PythonBroadcast import org.apache.spark.broadcast.HttpBroadcast import org.apache.spark.network.nio.{PutBlock, GotBlock, GetBlock} -import org.apache.spark.scheduler.MapStatus +import org.apache.spark.scheduler.{CompressedMapStatus, HighlyCompressedMapStatus} import org.apache.spark.storage._ import org.apache.spark.util.BoundedPriorityQueue import org.apache.spark.util.collection.CompactBuffer @@ -207,7 +207,8 @@ private[serializer] object KryoSerializer { classOf[PutBlock], classOf[GotBlock], classOf[GetBlock], - classOf[MapStatus], + classOf[CompressedMapStatus], + classOf[HighlyCompressedMapStatus], classOf[CompactBuffer[_]], classOf[BlockManagerId], classOf[Array[Byte]], |