aboutsummaryrefslogtreecommitdiff
path: root/streaming/pom.xml
diff options
context:
space:
mode:
authorlisurprise <zhichao.li@intel.com>2015-03-16 13:10:32 -0700
committerTathagata Das <tathagata.das1565@gmail.com>2015-03-16 13:10:32 -0700
commitf149b8b5e542af44650923d0156f037121b45a20 (patch)
tree80fdef3c31ad323a5d0ff3d6e9827c78045413b4 /streaming/pom.xml
parentd19efeddc0cb710c9496af11e447d39e1ad61b31 (diff)
downloadspark-f149b8b5e542af44650923d0156f037121b45a20.tar.gz
spark-f149b8b5e542af44650923d0156f037121b45a20.tar.bz2
spark-f149b8b5e542af44650923d0156f037121b45a20.zip
[SPARK-6077] Remove streaming tab while stopping StreamingContext
Currently we would create a new streaming tab for each streamingContext even if there's already one on the same sparkContext which would cause duplicate StreamingTab created and none of them is taking effect. snapshot: https://www.dropbox.com/s/t4gd6hqyqo0nivz/bad%20multiple%20streamings.png?dl=0 How to reproduce: 1) import org.apache.spark.SparkConf import org.apache.spark.streaming. {Seconds, StreamingContext} import org.apache.spark.storage.StorageLevel val ssc = new StreamingContext(sc, Seconds(1)) val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words = lines.flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() ssc.start() ..... 2) ssc.stop(false) val ssc = new StreamingContext(sc, Seconds(1)) val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words = lines.flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() ssc.start() Author: lisurprise <zhichao.li@intel.com> Closes #4828 from zhichao-li/master and squashes the following commits: c329806 [lisurprise] add test for attaching/detaching streaming tab 51e6c7f [lisurprise] move detach method into StreamingTab 31a44fa [lisurprise] add unit test for attaching and detaching new tab db25ed2 [lisurprise] clean code 8281bcb [lisurprise] clean code 193c542 [lisurprise] remove streaming tab while closing streaming context
Diffstat (limited to 'streaming/pom.xml')
-rw-r--r--streaming/pom.xml5
1 files changed, 5 insertions, 0 deletions
diff --git a/streaming/pom.xml b/streaming/pom.xml
index 0370b0e9e1..96508d83f4 100644
--- a/streaming/pom.xml
+++ b/streaming/pom.xml
@@ -83,6 +83,11 @@
<scope>test</scope>
</dependency>
<dependency>
+ <groupId>org.seleniumhq.selenium</groupId>
+ <artifactId>selenium-java</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>com.novocode</groupId>
<artifactId>junit-interface</artifactId>
<scope>test</scope>