aboutsummaryrefslogtreecommitdiff
path: root/docs/streaming-programming-guide.md
diff options
context:
space:
mode:
authorNeelesh Srinivas Salian <nsalian@cloudera.com>2015-06-18 09:44:36 -0700
committerDavies Liu <davies@databricks.com>2015-06-18 09:44:36 -0700
commitddc5baf17d7b09623b91190ee7754a6c8f7b5d10 (patch)
tree10eba4b5934d120b769d51425875d844726493a6 /docs/streaming-programming-guide.md
parente86fbdb1e6f1538f65ef78d90bbc41604f6bd580 (diff)
downloadspark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.tar.gz
spark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.tar.bz2
spark-ddc5baf17d7b09623b91190ee7754a6c8f7b5d10.zip
[SPARK-8320] [STREAMING] Add example in streaming programming guide that shows union of multiple input streams
Added python code to https://spark.apache.org/docs/latest/streaming-programming-guide.html to the Level of Parallelism in Data Receiving section. Please review and let me know if there are any additional changes that are needed. Thank you. Author: Neelesh Srinivas Salian <nsalian@cloudera.com> Closes #6862 from nssalian/SPARK-8320 and squashes the following commits: 4bfd126 [Neelesh Srinivas Salian] Changed loop structure to be more in line with Python style e5345de [Neelesh Srinivas Salian] Changes to kafak append, for loop and show to print() 3fc5c6d [Neelesh Srinivas Salian] SPARK-8320
Diffstat (limited to 'docs/streaming-programming-guide.md')
-rw-r--r--docs/streaming-programming-guide.md8
1 files changed, 8 insertions, 0 deletions
diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md
index 1eb3b30332..b784d59666 100644
--- a/docs/streaming-programming-guide.md
+++ b/docs/streaming-programming-guide.md
@@ -1937,6 +1937,14 @@ JavaPairDStream<String, String> unifiedStream = streamingContext.union(kafkaStre
unifiedStream.print();
{% endhighlight %}
</div>
+<div data-lang="python" markdown="1">
+{% highlight python %}
+numStreams = 5
+kafkaStreams = [KafkaUtils.createStream(...) for _ in range (numStreams)]
+unifiedStream = streamingContext.union(kafkaStreams)
+unifiedStream.print()
+{% endhighlight %}
+</div>
</div>
Another parameter that should be considered is the receiver's blocking interval,