diff options
| author | Bertrand Bossy <bertrandbossy@gmail.com> | 2014-09-14 21:10:17 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-09-14 21:10:17 -0700 |
| commit | c243b21a8ba2610266702e00d7d4b5443cb1f687 (patch) | |
| tree | 1bb73c12122225c31be670deaed747eef6026a07 /sql | |
| parent | 4e3fbe8cdb6c6291e219195abb272f3c81f0ed63 (diff) | |
| download | spark-c243b21a8ba2610266702e00d7d4b5443cb1f687.tar.gz spark-c243b21a8ba2610266702e00d7d4b5443cb1f687.tar.bz2 spark-c243b21a8ba2610266702e00d7d4b5443cb1f687.zip | |
SPARK-3039: Allow spark to be built using avro-mapred for hadoop2
SPARK-3039: Adds the maven property "avro.mapred.classifier" to build spark-assembly with avro-mapred with support for the new Hadoop API. Sets this property to hadoop2 for Hadoop 2 profiles.
I am not very familiar with maven, nor do I know whether this potentially breaks something in the hive part of spark. There might be a more elegant way of doing this.
Author: Bertrand Bossy <bertrandbossy@gmail.com>
Closes #1945 from bbossy/SPARK-3039 and squashes the following commits:
c32ce59 [Bertrand Bossy] SPARK-3039: Allow spark to be built using avro-mapred for hadoop2
Diffstat (limited to 'sql')
| -rw-r--r-- | sql/hive/pom.xml | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/sql/hive/pom.xml b/sql/hive/pom.xml index 45a4c6dc98..9d7a02bf7b 100644 --- a/sql/hive/pom.xml +++ b/sql/hive/pom.xml @@ -95,6 +95,15 @@ <dependency> <groupId>org.apache.avro</groupId> <artifactId>avro</artifactId> + <version>${avro.version}</version> + </dependency> + <!-- use the build matching the hadoop api of avro-mapred (i.e. no classifier for hadoop 1 API, + hadoop2 classifier for hadoop 2 API. avro-mapred is a dependency of org.spark-project.hive:hive-serde --> + <dependency> + <groupId>org.apache.avro</groupId> + <artifactId>avro-mapred</artifactId> + <version>${avro.version}</version> + <classifier>${avro.mapred.classifier}</classifier> </dependency> <dependency> <groupId>org.scalatest</groupId> |