diff options
author | WeichenXu <WeichenXu123@outlook.com> | 2016-07-23 12:32:30 +0100 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2016-07-23 12:32:30 +0100 |
commit | 25db51675f43048d61ced8221dcb4885cc5143c1 (patch) | |
tree | 7fab2d674faeb1b7f941a91f367f6f2b0e841329 /python/pyspark/sql/streaming.py | |
parent | e10b8741d86a0a625d28bcb1c654736a260be85e (diff) | |
download | spark-25db51675f43048d61ced8221dcb4885cc5143c1.tar.gz spark-25db51675f43048d61ced8221dcb4885cc5143c1.tar.bz2 spark-25db51675f43048d61ced8221dcb4885cc5143c1.zip |
[SPARK-16561][MLLIB] fix multivarOnlineSummary min/max bug
## What changes were proposed in this pull request?
renaming var names to make code more clear:
nnz => weightSum
weightSum => totalWeightSum
and add a new member vector `nnz` (not `nnz` in previous code, which renamed to `weightSum`) to count each dimensions non-zero value number.
using `nnz` which I added above instead of `weightSum` when calculating min/max so that it fix several numerical error in some extreme case.
## How was this patch tested?
A new testcase added.
Author: WeichenXu <WeichenXu123@outlook.com>
Closes #14216 from WeichenXu123/multivarOnlineSummary.
Diffstat (limited to 'python/pyspark/sql/streaming.py')
0 files changed, 0 insertions, 0 deletions