diff options
| author | Shixiong Zhu <shixiong@databricks.com> | 2016-02-04 12:43:16 -0800 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2016-02-04 12:43:16 -0800 |
| commit | 8e2f296306131e6c7c2f06d6672995d3ff8ab021 (patch) | |
| tree | c338de2d4c0e2cbf37c8f70d65e8a9b8f9e7a7b6 /streaming/src/test | |
| parent | bd38dd6f75c4af0f8f32bb21a82da53fffa5e825 (diff) | |
| download | spark-8e2f296306131e6c7c2f06d6672995d3ff8ab021.tar.gz spark-8e2f296306131e6c7c2f06d6672995d3ff8ab021.tar.bz2 spark-8e2f296306131e6c7c2f06d6672995d3ff8ab021.zip | |
[SPARK-13195][STREAMING] Fix NoSuchElementException when a state is not set but timeoutThreshold is defined
Check the state Existence before calling get.
Author: Shixiong Zhu <shixiong@databricks.com>
Closes #11081 from zsxwing/SPARK-13195.
Diffstat (limited to 'streaming/src/test')
| -rw-r--r-- | streaming/src/test/scala/org/apache/spark/streaming/rdd/MapWithStateRDDSuite.scala | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/streaming/src/test/scala/org/apache/spark/streaming/rdd/MapWithStateRDDSuite.scala b/streaming/src/test/scala/org/apache/spark/streaming/rdd/MapWithStateRDDSuite.scala index 5b13fd6ad6..e8c814ba71 100644 --- a/streaming/src/test/scala/org/apache/spark/streaming/rdd/MapWithStateRDDSuite.scala +++ b/streaming/src/test/scala/org/apache/spark/streaming/rdd/MapWithStateRDDSuite.scala @@ -190,6 +190,11 @@ class MapWithStateRDDSuite extends SparkFunSuite with RDDCheckpointTester with B timeoutThreshold = Some(initialTime + 1), removeTimedoutData = true, expectedStates = Nil, expectedTimingOutStates = Nil, expectedRemovedStates = Seq(123)) + // If a state is not set but timeoutThreshold is defined, we should ignore this state. + // Previously it threw NoSuchElementException (SPARK-13195). + assertRecordUpdate(initStates = Seq(), data = Seq("noop"), + timeoutThreshold = Some(initialTime + 1), removeTimedoutData = true, + expectedStates = Nil, expectedTimingOutStates = Nil) } test("states generated by MapWithStateRDD") { |