diff options
| author | Andrew Or <andrew@databricks.com> | 2014-11-13 11:54:45 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-11-13 11:54:45 -0800 |
| commit | a0fa1ba704355a82e168aa9c16ecfed30128ade0 (patch) | |
| tree | d7f0f02fd2e8433b549b47c263f6b77eb7e5c932 /make-distribution.sh | |
| parent | ca26a212fda39a15fde09dfdb2fbe69580a717f6 (diff) | |
| download | spark-a0fa1ba704355a82e168aa9c16ecfed30128ade0.tar.gz spark-a0fa1ba704355a82e168aa9c16ecfed30128ade0.tar.bz2 spark-a0fa1ba704355a82e168aa9c16ecfed30128ade0.zip | |
[HOT FIX] make-distribution.sh fails if Yarn shuffle jar DNE
This is introduced in #3147 and is failing builds without the `-Pyarn` profile.
Author: Andrew Or <andrew@databricks.com>
Closes #3250 from andrewor14/fix-yarn-shuffle-build and squashes the following commits:
42b3d37 [Andrew Or] Do not fail fast if Yarn shuffle jar does not exist
Diffstat (limited to 'make-distribution.sh')
| -rwxr-xr-x | make-distribution.sh | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/make-distribution.sh b/make-distribution.sh index 55cbdc14ac..2267b1aa08 100755 --- a/make-distribution.sh +++ b/make-distribution.sh @@ -181,7 +181,9 @@ echo "Spark $VERSION$GITREVSTRING built for Hadoop $SPARK_HADOOP_VERSION" > "$DI # Copy jars cp "$FWDIR"/assembly/target/scala*/*assembly*hadoop*.jar "$DISTDIR/lib/" cp "$FWDIR"/examples/target/scala*/spark-examples*.jar "$DISTDIR/lib/" -cp "$FWDIR"/network/yarn/target/scala*/spark-*-yarn-shuffle.jar "$DISTDIR/lib/" +# This will fail if the -Pyarn profile is not provided +# In this case, silence the error and ignore the return code of this command +cp "$FWDIR"/network/yarn/target/scala*/spark-*-yarn-shuffle.jar "$DISTDIR/lib/" &> /dev/null || : # Copy example sources (needed for python and SQL) mkdir -p "$DISTDIR/examples/src/main" |