diff options
| author | Shixiong Zhu <shixiong@databricks.com> | 2016-01-07 17:46:24 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-01-07 17:46:24 -0800 |
| commit | 28e0e500a2062baeda8c887e17dc8ab2b7d7d4b4 (patch) | |
| tree | ffe4125cab4f5520a2b1c4159c84c1d26cfc59a1 /project | |
| parent | c94199e977279d9b4658297e8108b46bdf30157b (diff) | |
| download | spark-28e0e500a2062baeda8c887e17dc8ab2b7d7d4b4.tar.gz spark-28e0e500a2062baeda8c887e17dc8ab2b7d7d4b4.tar.bz2 spark-28e0e500a2062baeda8c887e17dc8ab2b7d7d4b4.zip | |
[SPARK-12591][STREAMING] Register OpenHashMapBasedStateMap for Kryo
The default serializer in Kryo is FieldSerializer and it ignores transient fields and never calls `writeObject` or `readObject`. So we should register OpenHashMapBasedStateMap using `DefaultSerializer` to make it work with Kryo.
Author: Shixiong Zhu <shixiong@databricks.com>
Closes #10609 from zsxwing/SPARK-12591.
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 69e5bc881b..40559a0910 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -120,6 +120,10 @@ object MimaExcludes { ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.util.Vector$Multiplier"), ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.util.Vector$") ) ++ Seq( + // SPARK-12591 Register OpenHashMapBasedStateMap for Kryo + ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.serializer.KryoInputDataInputBridge"), + ProblemFilters.exclude[MissingClassProblem]("org.apache.spark.serializer.KryoOutputDataOutputBridge") + ) ++ Seq( // SPARK-12510 Refactor ActorReceiver to support Java ProblemFilters.exclude[AbstractClassProblem]("org.apache.spark.streaming.receiver.ActorReceiver") ) |