diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-05-04 12:22:23 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-05-04 12:22:23 -0700 |
| commit | 0c98a8f6a761b941a9e0cf6fde6df8d0b8d24057 (patch) | |
| tree | 23e422a365b2e98f0f8e86e3dbb51000449a8155 /bin | |
| parent | 0088cede592540f35f9aec0f24dc1d9bd690d878 (diff) | |
| download | spark-0c98a8f6a761b941a9e0cf6fde6df8d0b8d24057.tar.gz spark-0c98a8f6a761b941a9e0cf6fde6df8d0b8d24057.tar.bz2 spark-0c98a8f6a761b941a9e0cf6fde6df8d0b8d24057.zip | |
SPARK-1703 Warn users if Spark is run on JRE6 but compiled with JDK7.
This add some guards and good warning messages if users hit this issue. /cc @aarondav with whom I discussed parts of the design.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #627 from pwendell/jdk6 and squashes the following commits:
a38a958 [Patrick Wendell] Code review feedback
94e9f84 [Patrick Wendell] SPARK-1703 Warn users if Spark is run on JRE6 but compiled with JDK7.
Diffstat (limited to 'bin')
| -rwxr-xr-x | bin/compute-classpath.sh | 14 | ||||
| -rwxr-xr-x | bin/spark-class | 9 |
2 files changed, 22 insertions, 1 deletions
diff --git a/bin/compute-classpath.sh b/bin/compute-classpath.sh index 70ac71e45d..8dc547b379 100755 --- a/bin/compute-classpath.sh +++ b/bin/compute-classpath.sh @@ -32,6 +32,12 @@ CLASSPATH="$SPARK_CLASSPATH:$SPARK_SUBMIT_CLASSPATH:$FWDIR/conf" ASSEMBLY_DIR="$FWDIR/assembly/target/scala-$SCALA_VERSION" +if [ -n "${JAVA_HOME}" ]; then + JAR_CMD="${JAVA_HOME}/bin/jar" +else + JAR_CMD="jar" +fi + # First check if we have a dependencies jar. If so, include binary classes with the deps jar if [ -f "$ASSEMBLY_DIR"/spark-assembly*hadoop*-deps.jar ]; then CLASSPATH="$CLASSPATH:$FWDIR/core/target/scala-$SCALA_VERSION/classes" @@ -55,6 +61,14 @@ else else ASSEMBLY_JAR=`ls "$ASSEMBLY_DIR"/spark-assembly*hadoop*.jar` fi + jar_error_check=$($JAR_CMD -tf $ASSEMBLY_JAR org/apache/spark/SparkContext 2>&1) + if [[ "$jar_error_check" =~ "invalid CEN header" ]]; then + echo "Loading Spark jar with '$JAR_CMD' failed. " + echo "This is likely because Spark was compiled with Java 7 and run " + echo "with Java 6. (see SPARK-1703). Please use Java 7 to run Spark " + echo "or build Spark with Java 6." + exit 1 + fi CLASSPATH="$CLASSPATH:$ASSEMBLY_JAR" fi diff --git a/bin/spark-class b/bin/spark-class index e8160c8af6..72f8b9bf9a 100755 --- a/bin/spark-class +++ b/bin/spark-class @@ -138,7 +138,14 @@ if [ -e "$TOOLS_DIR"/target/spark-tools*[0-9Tg].jar ]; then fi # Compute classpath using external script -CLASSPATH=`$FWDIR/bin/compute-classpath.sh` +classpath_output=$($FWDIR/bin/compute-classpath.sh) +if [[ "$?" != "0" ]]; then + echo "$classpath_output" + exit 1 +else + CLASSPATH=$classpath_output +fi + if [[ "$1" =~ org.apache.spark.tools.* ]]; then CLASSPATH="$CLASSPATH:$SPARK_TOOLS_JAR" fi |