diff options
| author | Calvin Jia <jia.calvin@gmail.com> | 2015-04-24 17:57:41 -0400 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-04-24 17:57:41 -0400 |
| commit | 438859eb7c4e605bb4041d9a547a16be9c827c75 (patch) | |
| tree | 83b0589ea748fd090559532b3756c15ffbab6adb /examples/pom.xml | |
| parent | caf0136ec5838cf5bf61f39a5b3474a505a6ae11 (diff) | |
| download | spark-438859eb7c4e605bb4041d9a547a16be9c827c75.tar.gz spark-438859eb7c4e605bb4041d9a547a16be9c827c75.tar.bz2 spark-438859eb7c4e605bb4041d9a547a16be9c827c75.zip | |
[SPARK-6122] [CORE] Upgrade tachyon-client version to 0.6.3
This is a reopening of #4867.
A short summary of the issues resolved from the previous PR:
1. HTTPClient version mismatch: Selenium (used for UI tests) requires version 4.3.x, and Tachyon included 4.2.5 through a transitive dependency of its shaded thrift jar. To address this, Tachyon 0.6.3 will promote the transitive dependencies of the shaded jar so they can be excluded in spark.
2. Jackson-Mapper-ASL version mismatch: In lower versions of hadoop-client (ie. 1.0.4), version 1.0.1 is included. The parquet library used in spark sql requires version 1.8+. Its unclear to me why upgrading tachyon-client would cause this dependency to break. The solution was to exclude jackson-mapper-asl from hadoop-client.
It seems that the dependency management in spark-parent will not work on transitive dependencies, one way to make sure jackson-mapper-asl is included with the correct version is to add it as a top level dependency. The best solution would be to exclude the dependency in the modules which require a higher version, but that did not fix the unit tests. Any suggestions on the best way to solve this would be appreciated!
Author: Calvin Jia <jia.calvin@gmail.com>
Closes #5354 from calvinjia/upgrade_tachyon_0.6.3 and squashes the following commits:
0eefe4d [Calvin Jia] Handle httpclient version in maven dependency management. Remove httpclient version setting from profiles.
7c00dfa [Calvin Jia] Set httpclient version to 4.3.2 for selenium. Specify version of httpclient for sql/hive (previously 4.2.5 transitive dependency of libthrift).
9263097 [Calvin Jia] Merge master to test latest changes
dbfc1bd [Calvin Jia] Use Tachyon 0.6.4 for cleaner dependencies.
e2ff80a [Calvin Jia] Exclude the jetty and curator promoted dependencies from tachyon-client.
a3a29da [Calvin Jia] Update tachyon-client exclusions.
0ae6c97 [Calvin Jia] Change tachyon version to 0.6.3
a204df9 [Calvin Jia] Update make distribution tachyon version.
a93c94f [Calvin Jia] Exclude jackson-mapper-asl from hadoop client since it has a lower version than spark's expected version.
a8a923c [Calvin Jia] Exclude httpcomponents from Tachyon
910fabd [Calvin Jia] Update to master
eed9230 [Calvin Jia] Update tachyon version to 0.6.1.
11907b3 [Calvin Jia] Use TachyonURI for tachyon paths instead of strings.
71bf441 [Calvin Jia] Upgrade Tachyon client version to 0.6.0.
Diffstat (limited to 'examples/pom.xml')
| -rw-r--r-- | examples/pom.xml | 5 |
1 files changed, 0 insertions, 5 deletions
diff --git a/examples/pom.xml b/examples/pom.xml index afd7c6d52f..df1717403b 100644 --- a/examples/pom.xml +++ b/examples/pom.xml @@ -390,11 +390,6 @@ <artifactId>spark-streaming-kinesis-asl_${scala.binary.version}</artifactId> <version>${project.version}</version> </dependency> - <dependency> - <groupId>org.apache.httpcomponents</groupId> - <artifactId>httpclient</artifactId> - <version>${commons.httpclient.version}</version> - </dependency> </dependencies> </profile> <profile> |