diff options
author | Davies Liu <davies.liu@gmail.com> | 2014-10-18 19:14:48 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@databricks.com> | 2014-10-18 19:14:48 -0700 |
commit | 05db2da7dc256822cdb602c4821cbb9fb84dac98 (patch) | |
tree | d80715dff7672792539f9aaa66858999100324f5 /python/pyspark/streaming | |
parent | f406a8391825d8866110f29a0d656c82cd064520 (diff) | |
download | spark-05db2da7dc256822cdb602c4821cbb9fb84dac98.tar.gz spark-05db2da7dc256822cdb602c4821cbb9fb84dac98.tar.bz2 spark-05db2da7dc256822cdb602c4821cbb9fb84dac98.zip |
[SPARK-3952] [Streaming] [PySpark] add Python examples in Streaming Programming Guide
Having Python examples in Streaming Programming Guide.
Also add RecoverableNetworkWordCount example.
Author: Davies Liu <davies.liu@gmail.com>
Author: Davies Liu <davies@databricks.com>
Closes #2808 from davies/pyguide and squashes the following commits:
8d4bec4 [Davies Liu] update readme
26a7e37 [Davies Liu] fix format
3821c4d [Davies Liu] address comments, add missing file
7e4bb8a [Davies Liu] add Python examples in Streaming Programming Guide
Diffstat (limited to 'python/pyspark/streaming')
-rw-r--r-- | python/pyspark/streaming/dstream.py | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/python/pyspark/streaming/dstream.py b/python/pyspark/streaming/dstream.py index 5ae5cf07f0..0826ddc56e 100644 --- a/python/pyspark/streaming/dstream.py +++ b/python/pyspark/streaming/dstream.py @@ -441,9 +441,11 @@ class DStream(object): if `invReduceFunc` is not None, the reduction is done incrementally using the old window's reduced value : - 1. reduce the new values that entered the window (e.g., adding new counts) - 2. "inverse reduce" the old values that left the window (e.g., subtracting old counts) - This is more efficient than `invReduceFunc` is None. + + 1. reduce the new values that entered the window (e.g., adding new counts) + + 2. "inverse reduce" the old values that left the window (e.g., subtracting old counts) + This is more efficient than `invReduceFunc` is None. @param reduceFunc: associative reduce function @param invReduceFunc: inverse reduce function of `reduceFunc` |