diff options
| author | Andrew Or <andrew@databricks.com> | 2014-11-13 11:54:45 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-11-13 11:55:08 -0800 |
| commit | c07592e4050d7cc7c7288a4b9909cc28cd5467a3 (patch) | |
| tree | 6c847fb8d45c3a6978f8ba07070e103540a8bcf6 /make-distribution.sh | |
| parent | ff94283205d6b9774b700974fac0d4dfc33ef3e3 (diff) | |
| download | spark-c07592e4050d7cc7c7288a4b9909cc28cd5467a3.tar.gz spark-c07592e4050d7cc7c7288a4b9909cc28cd5467a3.tar.bz2 spark-c07592e4050d7cc7c7288a4b9909cc28cd5467a3.zip | |
[HOT FIX] make-distribution.sh fails if Yarn shuffle jar DNE
This is introduced in #3147 and is failing builds without the `-Pyarn` profile.
Author: Andrew Or <andrew@databricks.com>
Closes #3250 from andrewor14/fix-yarn-shuffle-build and squashes the following commits:
42b3d37 [Andrew Or] Do not fail fast if Yarn shuffle jar does not exist
(cherry picked from commit a0fa1ba704355a82e168aa9c16ecfed30128ade0)
Signed-off-by: Andrew Or <andrew@databricks.com>
Diffstat (limited to 'make-distribution.sh')
| -rwxr-xr-x | make-distribution.sh | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/make-distribution.sh b/make-distribution.sh index 55cbdc14ac..2267b1aa08 100755 --- a/make-distribution.sh +++ b/make-distribution.sh @@ -181,7 +181,9 @@ echo "Spark $VERSION$GITREVSTRING built for Hadoop $SPARK_HADOOP_VERSION" > "$DI # Copy jars cp "$FWDIR"/assembly/target/scala*/*assembly*hadoop*.jar "$DISTDIR/lib/" cp "$FWDIR"/examples/target/scala*/spark-examples*.jar "$DISTDIR/lib/" -cp "$FWDIR"/network/yarn/target/scala*/spark-*-yarn-shuffle.jar "$DISTDIR/lib/" +# This will fail if the -Pyarn profile is not provided +# In this case, silence the error and ignore the return code of this command +cp "$FWDIR"/network/yarn/target/scala*/spark-*-yarn-shuffle.jar "$DISTDIR/lib/" &> /dev/null || : # Copy example sources (needed for python and SQL) mkdir -p "$DISTDIR/examples/src/main" |