aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/streaming/dstream.py
diff options
context:
space:
mode:
authorDavies Liu <davies.liu@gmail.com>2014-10-18 19:14:48 -0700
committerJosh Rosen <joshrosen@databricks.com>2014-10-18 19:14:48 -0700
commit05db2da7dc256822cdb602c4821cbb9fb84dac98 (patch)
treed80715dff7672792539f9aaa66858999100324f5 /python/pyspark/streaming/dstream.py
parentf406a8391825d8866110f29a0d656c82cd064520 (diff)
downloadspark-05db2da7dc256822cdb602c4821cbb9fb84dac98.tar.gz
spark-05db2da7dc256822cdb602c4821cbb9fb84dac98.tar.bz2
spark-05db2da7dc256822cdb602c4821cbb9fb84dac98.zip
[SPARK-3952] [Streaming] [PySpark] add Python examples in Streaming Programming Guide
Having Python examples in Streaming Programming Guide. Also add RecoverableNetworkWordCount example. Author: Davies Liu <davies.liu@gmail.com> Author: Davies Liu <davies@databricks.com> Closes #2808 from davies/pyguide and squashes the following commits: 8d4bec4 [Davies Liu] update readme 26a7e37 [Davies Liu] fix format 3821c4d [Davies Liu] address comments, add missing file 7e4bb8a [Davies Liu] add Python examples in Streaming Programming Guide
Diffstat (limited to 'python/pyspark/streaming/dstream.py')
-rw-r--r--python/pyspark/streaming/dstream.py8
1 files changed, 5 insertions, 3 deletions
diff --git a/python/pyspark/streaming/dstream.py b/python/pyspark/streaming/dstream.py
index 5ae5cf07f0..0826ddc56e 100644
--- a/python/pyspark/streaming/dstream.py
+++ b/python/pyspark/streaming/dstream.py
@@ -441,9 +441,11 @@ class DStream(object):
if `invReduceFunc` is not None, the reduction is done incrementally
using the old window's reduced value :
- 1. reduce the new values that entered the window (e.g., adding new counts)
- 2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
- This is more efficient than `invReduceFunc` is None.
+
+ 1. reduce the new values that entered the window (e.g., adding new counts)
+
+ 2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
+ This is more efficient than `invReduceFunc` is None.
@param reduceFunc: associative reduce function
@param invReduceFunc: inverse reduce function of `reduceFunc`