diff options
| author | JihongMa <linlin200605@gmail.com> | 2015-09-12 10:17:15 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2015-09-12 10:17:15 -0700 |
| commit | f4a22808e03fa12bfe1bfc82cf713cfda7e063a9 (patch) | |
| tree | 49d22700542e44203793940eb28341e8df573cd5 /python | |
| parent | 22730ad54d681ad30e63fe910e8d89360853177d (diff) | |
| download | spark-f4a22808e03fa12bfe1bfc82cf713cfda7e063a9.tar.gz spark-f4a22808e03fa12bfe1bfc82cf713cfda7e063a9.tar.bz2 spark-f4a22808e03fa12bfe1bfc82cf713cfda7e063a9.zip | |
[SPARK-6548] Adding stddev to DataFrame functions
Adding STDDEV support for DataFrame using 1-pass online /parallel algorithm to compute variance. Please review the code change.
Author: JihongMa <linlin200605@gmail.com>
Author: Jihong MA <linlin200605@gmail.com>
Author: Jihong MA <jihongma@jihongs-mbp.usca.ibm.com>
Author: Jihong MA <jihongma@Jihongs-MacBook-Pro.local>
Closes #6297 from JihongMA/SPARK-SQL.
Diffstat (limited to 'python')
| -rw-r--r-- | python/pyspark/sql/dataframe.py | 36 |
1 files changed, 18 insertions, 18 deletions
diff --git a/python/pyspark/sql/dataframe.py b/python/pyspark/sql/dataframe.py index c5bf557912..fb995fa3a7 100644 --- a/python/pyspark/sql/dataframe.py +++ b/python/pyspark/sql/dataframe.py @@ -653,25 +653,25 @@ class DataFrame(object): guarantee about the backward compatibility of the schema of the resulting DataFrame. >>> df.describe().show() - +-------+---+ - |summary|age| - +-------+---+ - | count| 2| - | mean|3.5| - | stddev|1.5| - | min| 2| - | max| 5| - +-------+---+ + +-------+------------------+ + |summary| age| + +-------+------------------+ + | count| 2| + | mean| 3.5| + | stddev|2.1213203435596424| + | min| 2| + | max| 5| + +-------+------------------+ >>> df.describe(['age', 'name']).show() - +-------+---+-----+ - |summary|age| name| - +-------+---+-----+ - | count| 2| 2| - | mean|3.5| null| - | stddev|1.5| null| - | min| 2|Alice| - | max| 5| Bob| - +-------+---+-----+ + +-------+------------------+-----+ + |summary| age| name| + +-------+------------------+-----+ + | count| 2| 2| + | mean| 3.5| null| + | stddev|2.1213203435596424| null| + | min| 2|Alice| + | max| 5| Bob| + +-------+------------------+-----+ """ if len(cols) == 1 and isinstance(cols[0], list): cols = cols[0] |