diff options
author | Patrick Wendell <pwendell@gmail.com> | 2014-04-28 17:29:22 -0700 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-28 17:29:22 -0700 |
commit | 949e393101e19cd00591a9930c4b364278e22609 (patch) | |
tree | dcfa2c2337d25df917486f7f75771718a371055f /bin/spark-shell | |
parent | cae054aaf41ca0ee585231896db67169b61af689 (diff) | |
download | spark-949e393101e19cd00591a9930c4b364278e22609.tar.gz spark-949e393101e19cd00591a9930c4b364278e22609.tar.bz2 spark-949e393101e19cd00591a9930c4b364278e22609.zip |
SPARK-1654 and SPARK-1653: Fixes in spark-submit.
Deals with two issues:
1. Spark shell didn't correctly pass quoted arguments to spark-submit.
```./bin/spark-shell --driver-java-options "-Dfoo=f -Dbar=b"```
2. Spark submit used deprecated environment variables (SPARK_CLASSPATH)
which triggered warnings. Now we use new, more narrowly scoped,
variables.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #576 from pwendell/spark-submit and squashes the following commits:
67004c9 [Patrick Wendell] SPARK-1654 and SPARK-1653: Fixes in spark-submit.
Diffstat (limited to 'bin/spark-shell')
-rwxr-xr-x | bin/spark-shell | 11 |
1 files changed, 5 insertions, 6 deletions
diff --git a/bin/spark-shell b/bin/spark-shell index f1f3c18877..7f03349c5e 100755 --- a/bin/spark-shell +++ b/bin/spark-shell @@ -20,7 +20,6 @@ # # Shell script for starting the Spark Shell REPL -args="$@" cygwin=false case "`uname`" in CYGWIN*) cygwin=true;; @@ -46,12 +45,12 @@ function main(){ # "Backspace sends ^H" setting in "Keys" section of the Mintty options # (see https://github.com/sbt/sbt/issues/562). stty -icanon min 1 -echo > /dev/null 2>&1 - export SPARK_REPL_OPTS="$SPARK_REPL_OPTS -Djline.terminal=unix" - $FWDIR/bin/spark-submit spark-internal "$args" --class org.apache.spark.repl.Main + export SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Djline.terminal=unix" + $FWDIR/bin/spark-submit spark-internal "$@" --class org.apache.spark.repl.Main stty icanon echo > /dev/null 2>&1 else - export SPARK_REPL_OPTS - $FWDIR/bin/spark-submit spark-internal "$args" --class org.apache.spark.repl.Main + export SPARK_SUBMIT_OPTS + $FWDIR/bin/spark-submit spark-internal "$@" --class org.apache.spark.repl.Main fi } @@ -83,7 +82,7 @@ if [[ ! $? ]]; then saved_stty="" fi -main +main "$@" # record the exit status lest it be overwritten: # then reenable echo and propagate the code. |