diff options
author | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-07-13 19:10:00 -0700 |
---|---|---|
committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-07-13 19:10:00 -0700 |
commit | cd28d9c14732a91416589bb3dc523533495240e8 (patch) | |
tree | 1ab921af65b56d3c3d5dd3a94670bda8d8f68a36 /make-distribution.sh | |
parent | 1889f9f450d3e28864a5ac7b5f9d3831cf7b17bf (diff) | |
parent | 1107b4d55b9d5845ebf49fb2eef8cde55095982a (diff) | |
download | spark-cd28d9c14732a91416589bb3dc523533495240e8.tar.gz spark-cd28d9c14732a91416589bb3dc523533495240e8.tar.bz2 spark-cd28d9c14732a91416589bb3dc523533495240e8.zip |
Merge remote-tracking branch 'origin/pr/662'
Conflicts:
bin/compute-classpath.sh
Diffstat (limited to 'make-distribution.sh')
-rwxr-xr-x | make-distribution.sh | 39 |
1 files changed, 39 insertions, 0 deletions
diff --git a/make-distribution.sh b/make-distribution.sh new file mode 100755 index 0000000000..feb13d52f9 --- /dev/null +++ b/make-distribution.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# +# Script to create a binary distribution for easy deploys of Spark. +# The distribution directory defaults to dist/ but can be overridden below. +# The distribution contains fat (assembly) jars that include the Scala library, +# so it is completely self contained. +# It does not contain source or *.class files. +# +# Recommended deploy/testing procedure (standalone mode): +# 1) Rsync / deploy the dist/ dir to one host +# 2) cd to deploy dir; ./bin/start-master.sh +# 3) Verify master is up by visiting web page, ie http://master-ip:8080. Note the spark:// URL. +# 4) ./bin/start-slave.sh 1 <<spark:// URL>> +# 5) MASTER="spark://my-master-ip:7077" ./spark-shell + +# Figure out where the Spark framework is installed +FWDIR="$(cd `dirname $0`; pwd)" +DISTDIR="$FWDIR/dist" + +# Get version from SBT +export TERM=dumb # Prevents color codes in SBT output +VERSION=$($FWDIR/sbt/sbt "show version" | tail -1 | cut -f 2) +echo "Making distribution for Spark $VERSION in $DISTDIR..." + +# Build fat JAR +$FWDIR/sbt/sbt "repl/assembly" + +# Make directories +rm -rf "$DISTDIR" +mkdir -p "$DISTDIR/jars" +echo "$VERSION" >$DISTDIR/RELEASE + +# Copy jars +cp $FWDIR/repl/target/*.jar "$DISTDIR/jars/" + +# Copy other things +cp -r "$FWDIR/bin" "$DISTDIR" +cp -r "$FWDIR/conf" "$DISTDIR" +cp "$FWDIR/run" "$FWDIR/spark-shell" "$DISTDIR" |