diff options
author | Iulian Dragos <jaguarul@gmail.com> | 2015-02-02 14:00:33 -0800 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-02-02 14:00:33 -0800 |
commit | e908322cd5991e6cbdaaafb8cd494759dac01225 (patch) | |
tree | 66ce728fd4de2216dea0009eac5fc5c783a1cec5 /python/pyspark/context.py | |
parent | 683e938242e29a0d584452e5230b4168b85bdab2 (diff) | |
download | spark-e908322cd5991e6cbdaaafb8cd494759dac01225.tar.gz spark-e908322cd5991e6cbdaaafb8cd494759dac01225.tar.bz2 spark-e908322cd5991e6cbdaaafb8cd494759dac01225.zip |
[SPARK-4631][streaming][FIX] Wait for a receiver to start before publishing test data.
This fixes two sources of non-deterministic failures in this test:
- wait for a receiver to be up before pushing data through MQTT
- gracefully handle the case where the MQTT client is overloaded. There’s
a hard-coded limit of 10 in-flight messages, and this test may hit it.
Instead of crashing, we retry sending the message.
Both of these are needed to make the test pass reliably on my machine.
Author: Iulian Dragos <jaguarul@gmail.com>
Closes #4270 from dragos/issue/fix-flaky-test-SPARK-4631 and squashes the following commits:
f66c482 [Iulian Dragos] [SPARK-4631][streaming] Wait for a receiver to start before publishing test data.
d408a8e [Iulian Dragos] Install callback before connecting to MQTT broker.
Diffstat (limited to 'python/pyspark/context.py')
0 files changed, 0 insertions, 0 deletions