diff options
| author | Xiangrui Meng <meng@databricks.com> | 2014-11-01 15:21:36 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-11-01 15:21:36 -0700 |
| commit | d8176b1c2f22247ee724041aefa1af9118cf861d (patch) | |
| tree | 964064cf27df71c69d3e4e8a63acce9d444fd2ec /pom.xml | |
| parent | 7894de276b8d0b0e4efc654d0b254fc2a6f6077c (diff) | |
| download | spark-d8176b1c2f22247ee724041aefa1af9118cf861d.tar.gz spark-d8176b1c2f22247ee724041aefa1af9118cf861d.tar.bz2 spark-d8176b1c2f22247ee724041aefa1af9118cf861d.zip | |
[SPARK-4121] Set commons-math3 version based on hadoop profiles, instead of shading
In #2928 , we shade commons-math3 to prevent future conflicts with hadoop. It caused problems with our Jenkins master build with maven. Some tests used local-cluster mode, where the assembly jar contains relocated math3 classes, while mllib test code still compiles with core and the untouched math3 classes.
This PR sets commons-math3 version based on hadoop profiles.
pwendell JoshRosen srowen
Author: Xiangrui Meng <meng@databricks.com>
Closes #3023 from mengxr/SPARK-4121-alt and squashes the following commits:
580f6d9 [Xiangrui Meng] replace tab by spaces
7f71f08 [Xiangrui Meng] revert changes to PoissonSampler to avoid conflicts
d3353d9 [Xiangrui Meng] do not shade commons-math3
b4180dc [Xiangrui Meng] temp work
Diffstat (limited to 'pom.xml')
| -rw-r--r-- | pom.xml | 9 |
1 files changed, 8 insertions, 1 deletions
@@ -145,6 +145,7 @@ <aws.java.sdk.version>1.8.3</aws.java.sdk.version> <aws.kinesis.client.version>1.1.0</aws.kinesis.client.version> <commons.httpclient.version>4.2.6</commons.httpclient.version> + <commons.math3.version>3.1.1</commons.math3.version> <PermGen>64m</PermGen> <MaxPermGen>512m</MaxPermGen> @@ -318,7 +319,7 @@ <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-math3</artifactId> - <version>3.3</version> + <version>${commons.math3.version}</version> </dependency> <dependency> <groupId>com.google.code.findbugs</groupId> @@ -1175,6 +1176,10 @@ <!-- A series of build profiles where customizations for particular Hadoop releases can be made --> + <!-- Hadoop-a.b.c dependencies can be found at + http://hadoop.apache.org/docs/ra.b.c/hadoop-project-dist/hadoop-common/dependency-analysis.html + --> + <profile> <id>hadoop-0.23</id> <!-- SPARK-1121: Adds an explicit dependency on Avro to work around a Hadoop 0.23.X issue --> @@ -1204,6 +1209,7 @@ <hadoop.version>2.3.0</hadoop.version> <protobuf.version>2.5.0</protobuf.version> <jets3t.version>0.9.0</jets3t.version> + <commons.math3.version>3.1.1</commons.math3.version> <avro.mapred.classifier>hadoop2</avro.mapred.classifier> </properties> </profile> @@ -1214,6 +1220,7 @@ <hadoop.version>2.4.0</hadoop.version> <protobuf.version>2.5.0</protobuf.version> <jets3t.version>0.9.0</jets3t.version> + <commons.math3.version>3.1.1</commons.math3.version> <avro.mapred.classifier>hadoop2</avro.mapred.classifier> </properties> </profile> |