aboutsummaryrefslogtreecommitdiff
path: root/build
diff options
context:
space:
mode:
authorlisurprise <zhichao.li@intel.com>2015-03-16 13:10:32 -0700
committerTathagata Das <tathagata.das1565@gmail.com>2015-03-16 13:10:32 -0700
commitf149b8b5e542af44650923d0156f037121b45a20 (patch)
tree80fdef3c31ad323a5d0ff3d6e9827c78045413b4 /build
parentd19efeddc0cb710c9496af11e447d39e1ad61b31 (diff)
downloadspark-f149b8b5e542af44650923d0156f037121b45a20.tar.gz
spark-f149b8b5e542af44650923d0156f037121b45a20.tar.bz2
spark-f149b8b5e542af44650923d0156f037121b45a20.zip
[SPARK-6077] Remove streaming tab while stopping StreamingContext
Currently we would create a new streaming tab for each streamingContext even if there's already one on the same sparkContext which would cause duplicate StreamingTab created and none of them is taking effect. snapshot: https://www.dropbox.com/s/t4gd6hqyqo0nivz/bad%20multiple%20streamings.png?dl=0 How to reproduce: 1) import org.apache.spark.SparkConf import org.apache.spark.streaming. {Seconds, StreamingContext} import org.apache.spark.storage.StorageLevel val ssc = new StreamingContext(sc, Seconds(1)) val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words = lines.flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() ssc.start() ..... 2) ssc.stop(false) val ssc = new StreamingContext(sc, Seconds(1)) val lines = ssc.socketTextStream("localhost", 9999, StorageLevel.MEMORY_AND_DISK_SER) val words = lines.flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) wordCounts.print() ssc.start() Author: lisurprise <zhichao.li@intel.com> Closes #4828 from zhichao-li/master and squashes the following commits: c329806 [lisurprise] add test for attaching/detaching streaming tab 51e6c7f [lisurprise] move detach method into StreamingTab 31a44fa [lisurprise] add unit test for attaching and detaching new tab db25ed2 [lisurprise] clean code 8281bcb [lisurprise] clean code 193c542 [lisurprise] remove streaming tab while closing streaming context
Diffstat (limited to 'build')
0 files changed, 0 insertions, 0 deletions