diff options
author | Yanbo Liang <ybliang8@gmail.com> | 2016-07-19 12:31:04 +0100 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2016-07-19 12:31:04 +0100 |
commit | 670891496a82538a5e2bf981a4044fb6f4cbb062 (patch) | |
tree | a91eb7ad9962b2eb5f2de12e3522ec45aab951d1 /mllib/src/test/java/org | |
parent | 5d92326be76cb15edc6e18e94a373e197f696803 (diff) | |
download | spark-670891496a82538a5e2bf981a4044fb6f4cbb062.tar.gz spark-670891496a82538a5e2bf981a4044fb6f4cbb062.tar.bz2 spark-670891496a82538a5e2bf981a4044fb6f4cbb062.zip |
[SPARK-16494][ML] Upgrade breeze version to 0.12
## What changes were proposed in this pull request?
breeze 0.12 has been released for more than half a year, and it brings lots of new features, performance improvement and bug fixes.
One of the biggest features is ```LBFGS-B``` which is an implementation of ```LBFGS``` with box constraints and much faster for some special case.
We would like to implement Huber loss function for ```LinearRegression``` ([SPARK-3181](https://issues.apache.org/jira/browse/SPARK-3181)) and it requires ```LBFGS-B``` as the optimization solver. So we should bump up the dependent breeze version to 0.12.
For more features, improvements and bug fixes of breeze 0.12, you can refer the following link:
https://groups.google.com/forum/#!topic/scala-breeze/nEeRi_DcY5c
## How was this patch tested?
No new tests, should pass the existing ones.
Author: Yanbo Liang <ybliang8@gmail.com>
Closes #14150 from yanboliang/spark-16494.
Diffstat (limited to 'mllib/src/test/java/org')
-rw-r--r-- | mllib/src/test/java/org/apache/spark/ml/feature/JavaPCASuite.java | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/mllib/src/test/java/org/apache/spark/ml/feature/JavaPCASuite.java b/mllib/src/test/java/org/apache/spark/ml/feature/JavaPCASuite.java index ac479c0841..8c0338e284 100644 --- a/mllib/src/test/java/org/apache/spark/ml/feature/JavaPCASuite.java +++ b/mllib/src/test/java/org/apache/spark/ml/feature/JavaPCASuite.java @@ -107,7 +107,11 @@ public class JavaPCASuite extends SharedSparkSession { .fit(df); List<Row> result = pca.transform(df).select("pca_features", "expected").toJavaRDD().collect(); for (Row r : result) { - Assert.assertEquals(r.get(1), r.get(0)); + Vector calculatedVector = (Vector) r.get(0); + Vector expectedVector = (Vector) r.get(1); + for (int i = 0; i < calculatedVector.size(); i++) { + Assert.assertEquals(calculatedVector.apply(i), expectedVector.apply(i), 1.0e-8); + } } } } |