diff options
author | mbittmann <mbittmann@gmail.com> | 2015-02-08 10:13:29 +0000 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2015-02-08 10:13:29 +0000 |
commit | 48783136958e76d96f477802805e000ee5da5697 (patch) | |
tree | fdf86be431b6e715b678215bfd6df3f526d8519c /mllib | |
parent | 6fb141e2a9e728499f8782310560bfaef7a5ed6c (diff) | |
download | spark-48783136958e76d96f477802805e000ee5da5697.tar.gz spark-48783136958e76d96f477802805e000ee5da5697.tar.bz2 spark-48783136958e76d96f477802805e000ee5da5697.zip |
[SPARK-5656] Fail gracefully for large values of k and/or n that will ex...
...ceed max int.
Large values of k and/or n in EigenValueDecomposition.symmetricEigs will result in array initialization to a value larger than Integer.MAX_VALUE in the following: var v = new Array[Double](n * ncv)
Author: mbittmann <mbittmann@gmail.com>
Author: bittmannm <mark.bittmann@agilex.com>
Closes #4433 from mbittmann/master and squashes the following commits:
ee56e05 [mbittmann] [SPARK-5656] Combine checks into simple message
e49cbbb [mbittmann] [SPARK-5656] Simply error message
860836b [mbittmann] Array size check updates based on code review
a604816 [bittmannm] [SPARK-5656] Fail gracefully for large values of k and/or n that will exceed max int.
Diffstat (limited to 'mllib')
-rw-r--r-- | mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala b/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala index 3515461b52..9d6f975281 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala @@ -79,6 +79,9 @@ private[mllib] object EigenValueDecomposition { // Mode 1: A*x = lambda*x, A symmetric iparam(6) = 1 + require(n * ncv.toLong <= Integer.MAX_VALUE && ncv * (ncv.toLong + 8) <= Integer.MAX_VALUE, + s"k = $k and/or n = $n are too large to compute an eigendecomposition") + var ido = new intW(0) var info = new intW(0) var resid = new Array[Double](n) |