diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2014-09-12 14:54:42 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-09-12 14:54:42 -0700 |
| commit | af2583826c15d2a4e2732017ea20feeff0fb79f6 (patch) | |
| tree | 0d25121785197f37973e46c965c8471955761d56 /core/pom.xml | |
| parent | 1d767967e925f1d727957c2d43383ef6ad2c5d5e (diff) | |
| download | spark-af2583826c15d2a4e2732017ea20feeff0fb79f6.tar.gz spark-af2583826c15d2a4e2732017ea20feeff0fb79f6.tar.bz2 spark-af2583826c15d2a4e2732017ea20feeff0fb79f6.zip | |
[SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set.
When that option is used, the compiled classes from the build directory
are prepended to the classpath. Now that we avoid packaging Guava, that
means we have classes referencing the original Guava location in the app's
classpath, so errors happen.
For that case, add Guava manually to the classpath.
Note: if Spark is compiled with "-Phadoop-provided", it's tricky to
make things work with SPARK_PREPEND_CLASSES, because you need to add
the Hadoop classpath using SPARK_CLASSPATH and that means the older
Hadoop Guava overrides the newer one Spark needs. So someone using
SPARK_PREPEND_CLASSES needs to remember to not use that profile.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #2141 from vanzin/SPARK-3217 and squashes the following commits:
b967324 [Marcelo Vanzin] [SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set.
Diffstat (limited to 'core/pom.xml')
| -rw-r--r-- | core/pom.xml | 27 |
1 files changed, 27 insertions, 0 deletions
diff --git a/core/pom.xml b/core/pom.xml index b2b788a4bc..2a81f6df28 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -351,6 +351,33 @@ </execution> </executions> </plugin> + <!-- + Copy guava to the build directory. This is needed to make the SPARK_PREPEND_CLASSES + option work in compute-classpath.sh, since it would put the non-shaded Spark classes in + the runtime classpath. + --> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-dependency-plugin</artifactId> + <executions> + <execution> + <id>copy-dependencies</id> + <phase>package</phase> + <goals> + <goal>copy-dependencies</goal> + </goals> + <configuration> + <outputDirectory>${project.build.directory}</outputDirectory> + <overWriteReleases>false</overWriteReleases> + <overWriteSnapshots>false</overWriteSnapshots> + <overWriteIfNewer>true</overWriteIfNewer> + <useSubDirectoryPerType>true</useSubDirectoryPerType> + <includeArtifactIds>guava</includeArtifactIds> + <silent>true</silent> + </configuration> + </execution> + </executions> + </plugin> </plugins> <resources> |