diff options
| author | Shixiong Zhu <shixiong@databricks.com> | 2016-05-10 13:26:53 -0700 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2016-05-10 13:26:53 -0700 |
| commit | 9533f5390a3ad7ab96a7bea01cdb6aed89503a51 (patch) | |
| tree | caef694bf7758ba3138fb7232f052e3e5104e5da /python | |
| parent | 603c4f8ebde3ff61ac84a9460e6602c38a176e9f (diff) | |
| download | spark-9533f5390a3ad7ab96a7bea01cdb6aed89503a51.tar.gz spark-9533f5390a3ad7ab96a7bea01cdb6aed89503a51.tar.bz2 spark-9533f5390a3ad7ab96a7bea01cdb6aed89503a51.zip | |
[SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery
## What changes were proposed in this pull request?
Because this test extracts data from `DStream.generatedRDDs` before stopping, it may get data before checkpointing. Then after recovering from the checkpoint, `recoveredOffsetRanges` may contain something not in `offsetRangesBeforeStop`, which will fail the test. Adding `Thread.sleep(1000)` before `ssc.stop()` will reproduce this failure.
This PR just moves the logic of `offsetRangesBeforeStop` (also renamed to `offsetRangesAfterStop`) after `ssc.stop()` to fix the flaky test.
## How was this patch tested?
Jenkins unit tests.
Author: Shixiong Zhu <shixiong@databricks.com>
Closes #12903 from zsxwing/SPARK-6005.
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions