diff options
author | Neelesh Srinivas Salian <nsalian@cloudera.com> | 2015-06-18 09:44:36 -0700 |
---|---|---|
committer | Davies Liu <davies@databricks.com> | 2015-06-18 09:44:36 -0700 |
commit | ddc5baf17d7b09623b91190ee7754a6c8f7b5d10 (patch) | |
tree | 10eba4b5934d120b769d51425875d844726493a6 /docs/streaming-programming-guide.md | |
parent | e86fbdb1e6f1538f65ef78d90bbc41604f6bd580 (diff) | |
download | spark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.tar.gz spark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.tar.bz2 spark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.zip |
[SPARK-8320] [STREAMING] Add example in streaming programming guide that shows union of multiple input streams
Added python code to https://spark.apache.org/docs/latest/streaming-programming-guide.html
to the Level of Parallelism in Data Receiving section.
Please review and let me know if there are any additional changes that are needed.
Thank you.
Author: Neelesh Srinivas Salian <nsalian@cloudera.com>
Closes #6862 from nssalian/SPARK-8320 and squashes the following commits:
4bfd126 [Neelesh Srinivas Salian] Changed loop structure to be more in line with Python style
e5345de [Neelesh Srinivas Salian] Changes to kafak append, for loop and show to print()
3fc5c6d [Neelesh Srinivas Salian] SPARK-8320
Diffstat (limited to 'docs/streaming-programming-guide.md')
-rw-r--r-- | docs/streaming-programming-guide.md | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md index 1eb3b30332..b784d59666 100644 --- a/docs/streaming-programming-guide.md +++ b/docs/streaming-programming-guide.md @@ -1937,6 +1937,14 @@ JavaPairDStream<String, String> unifiedStream = streamingContext.union(kafkaStre unifiedStream.print(); {% endhighlight %} </div> +<div data-lang="python" markdown="1"> +{% highlight python %} +numStreams = 5 +kafkaStreams = [KafkaUtils.createStream(...) for _ in range (numStreams)] +unifiedStream = streamingContext.union(kafkaStreams) +unifiedStream.print() +{% endhighlight %} +</div> </div> Another parameter that should be considered is the receiver's blocking interval, |