diff options
| author | Xiangrui Meng <meng@databricks.com> | 2015-05-12 14:24:26 -0700 |
|---|---|---|
| committer | Joseph K. Bradley <joseph@databricks.com> | 2015-05-12 14:24:26 -0700 |
| commit | 23b9863e2aa7ecd0c4fa3aa8a59fdae09b4fe1d7 (patch) | |
| tree | 90683222288bbb083b31ea1e1e2fc61f1d9649ed /sql/core/src | |
| parent | 2a41c0d71a13558f12c6811bf98791e01186f3ad (diff) | |
| download | spark-23b9863e2aa7ecd0c4fa3aa8a59fdae09b4fe1d7.tar.gz spark-23b9863e2aa7ecd0c4fa3aa8a59fdae09b4fe1d7.tar.bz2 spark-23b9863e2aa7ecd0c4fa3aa8a59fdae09b4fe1d7.zip | |
[SPARK-7559] [MLLIB] Bucketizer should include the right most boundary in the last bucket.
We make special treatment for +inf in `Bucketizer`. This could be simplified by always including the largest split value in the last bucket. E.g., (x1, x2, x3) defines buckets [x1, x2) and [x2, x3]. This shouldn't affect user code much, and there are applications that need to include the right-most value. For example, we can bucketize ratings from 0 to 10 to bad, neutral, and good with splits 0, 4, 6, 10. It may reads weird if the users need to put 0, 4, 6, 10.1 (or 11).
This also update the impl to use `Arrays.binarySearch` and `withClue` in test.
yinxusen jkbradley
Author: Xiangrui Meng <meng@databricks.com>
Closes #6075 from mengxr/SPARK-7559 and squashes the following commits:
e28f910 [Xiangrui Meng] update bucketizer impl
Diffstat (limited to 'sql/core/src')
0 files changed, 0 insertions, 0 deletions