diff options
author | jerryshao <saisai.shao@intel.com> | 2015-01-22 21:58:53 -0800 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-01-22 21:59:22 -0800 |
commit | 5aaf0e0ff5e5082c0064f5f4065cd66a62aa72d6 (patch) | |
tree | 8d12d05873c28a9bc691ba459bd0c324886112e5 /examples | |
parent | 5d07488adc074ff8d5a10980dfc25cd2d33d0cf0 (diff) | |
download | spark-5aaf0e0ff5e5082c0064f5f4065cd66a62aa72d6.tar.gz spark-5aaf0e0ff5e5082c0064f5f4065cd66a62aa72d6.tar.bz2 spark-5aaf0e0ff5e5082c0064f5f4065cd66a62aa72d6.zip |
[SPARK-5233][Streaming] Fix error replaying of WAL introduced bug
Because of lacking of `BlockAllocationEvent` in WAL recovery, the dangled event will mix into the new batch, which will lead to the wrong result. Details can be seen in [SPARK-5233](https://issues.apache.org/jira/browse/SPARK-5233).
Author: jerryshao <saisai.shao@intel.com>
Closes #4032 from jerryshao/SPARK-5233 and squashes the following commits:
f0b0c0b [jerryshao] Further address the comments
a237c75 [jerryshao] Address the comments
e356258 [jerryshao] Fix bug in unit test
558bdc3 [jerryshao] Correctly replay the WAL log when recovering from failure
(cherry picked from commit 3c3fa632e6ba45ce536065aa1145698385301fb2)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
Diffstat (limited to 'examples')
-rw-r--r-- | examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala b/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala index c9e1511278..82aeaaf035 100644 --- a/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala +++ b/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala @@ -77,7 +77,7 @@ object KafkaWordCountProducer { val Array(brokers, topic, messagesPerSec, wordsPerMessage) = args - // Zookeper connection properties + // Zookeeper connection properties val props = new Properties() props.put("metadata.broker.list", brokers) props.put("serializer.class", "kafka.serializer.StringEncoder") |