diff options
| author | Marcin Tustin <mtustin@handybook.com> | 2016-04-03 17:42:33 -0700 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-04-03 17:42:33 -0700 |
| commit | 9023015f059327b3ce4a7eaf71e57ac77b84ad7b (patch) | |
| tree | 6f1b7d71a36f8acc573a9e8cce31ddd05efa50fb /common/network-shuffle/src | |
| parent | c238cd07448f94bbceb661daad90b6a6d597a846 (diff) | |
| download | spark-9023015f059327b3ce4a7eaf71e57ac77b84ad7b.tar.gz spark-9023015f059327b3ce4a7eaf71e57ac77b84ad7b.tar.bz2 spark-9023015f059327b3ce4a7eaf71e57ac77b84ad7b.zip | |
[SPARK-14163][CORE] SumEvaluator and countApprox cannot reliably handle RDDs of size 1
## What changes were proposed in this pull request?
This special cases 0 and 1 counts to avoid passing 0 degrees of freedom.
## How was this patch tested?
Tests run successfully. New test added.
## Note:
This recreates #11982 which was closed to due to non-updated diff. rxin srowen Commented there.
This also adds tests, reworks the code to perform the special casing (based on srowen's comments), and adds equality machinery for BoundedDouble, as well as changing how it is transformed to string.
Author: Marcin Tustin <mtustin@handybook.com>
Author: Marcin Tustin <mtustin@handy.com>
Closes #12016 from mtustin-handy/SPARK-14163.
Diffstat (limited to 'common/network-shuffle/src')
0 files changed, 0 insertions, 0 deletions