diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-01-22 15:45:04 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-01-22 15:45:04 -0800 |
| commit | 3184facdc5b1e9ded89133f9b1e4985c9ac78c55 (patch) | |
| tree | aecf92375025a4f9a09001aecbe9844b5b6e56fd /graphx | |
| parent | a1238bb5fcab763d32c729ea7ed99cb3c05c896f (diff) | |
| parent | 4476398f7dae1ea87d2b99cb60306cb3c221992f (diff) | |
| download | spark-3184facdc5b1e9ded89133f9b1e4985c9ac78c55.tar.gz spark-3184facdc5b1e9ded89133f9b1e4985c9ac78c55.tar.bz2 spark-3184facdc5b1e9ded89133f9b1e4985c9ac78c55.zip | |
Merge pull request #495 from srowen/GraphXCommonsMathDependency
Fix graphx Commons Math dependency
`graphx` depends on Commons Math (2.x) in `SVDPlusPlus.scala`. However the module doesn't declare this dependency. It happens to work because it is included by Hadoop artifacts. But, I can tell you this isn't true as of a month or so ago. Building versus recent Hadoop would fail. (That's how we noticed.)
The simple fix is to declare the dependency, as it should be. But it's also worth noting that `commons-math` is the old-ish 2.x line, while `commons-math3` is where newer 3.x releases are. Drop-in replacement, but different artifact and package name. Changing this only usage to `commons-math3` works, tests pass, and isn't surprising that it does, so is probably also worth changing. (A comment in some test code also references `commons-math3`, FWIW.)
It does raise another question though: `mllib` looks like it uses the `jblas` `DoubleMatrix` for general purpose vector/matrix stuff. Should `graphx` really use Commons Math for this? Beyond the tiny scope here but worth asking.
Diffstat (limited to 'graphx')
| -rw-r--r-- | graphx/pom.xml | 5 | ||||
| -rw-r--r-- | graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala | 2 |
2 files changed, 6 insertions, 1 deletions
diff --git a/graphx/pom.xml b/graphx/pom.xml index 4eca4747ea..d97dbb804b 100644 --- a/graphx/pom.xml +++ b/graphx/pom.xml @@ -38,6 +38,11 @@ <version>${project.version}</version> </dependency> <dependency> + <groupId>org.apache.commons</groupId> + <artifactId>commons-math3</artifactId> + <version>3.2</version> + </dependency> + <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-server</artifactId> </dependency> diff --git a/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala b/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala index c327ce7935..79280f836f 100644 --- a/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala +++ b/graphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala @@ -18,7 +18,7 @@ package org.apache.spark.graphx.lib import scala.util.Random -import org.apache.commons.math.linear._ +import org.apache.commons.math3.linear._ import org.apache.spark.rdd._ import org.apache.spark.graphx._ |