diff options
author | Reynold Xin <rxin@apache.org> | 2014-01-20 22:35:45 -0800 |
---|---|---|
committer | Reynold Xin <rxin@apache.org> | 2014-01-20 22:35:45 -0800 |
commit | 6b4eed779bd1889543ac2c058745bd0864f02b2a (patch) | |
tree | 38405d8f2bb3012e5b7efa91d8a933a0d28b6f8d | |
parent | 0367981d47761cdccd8a44fc6fe803079979c5e3 (diff) | |
parent | 8400536456ecff26145244cf74b7c00dd1c7034b (diff) | |
download | spark-6b4eed779bd1889543ac2c058745bd0864f02b2a.tar.gz spark-6b4eed779bd1889543ac2c058745bd0864f02b2a.tar.bz2 spark-6b4eed779bd1889543ac2c058745bd0864f02b2a.zip |
Merge pull request #449 from CrazyJvm/master
SPARK-1028 : fix "set MASTER automatically fails" bug.
spark-shell intends to set MASTER automatically if we do not provide the option when we start the shell , but there's a problem.
The condition is "if [[ "x" != "x$SPARK_MASTER_IP" && "y" != "y$SPARK_MASTER_PORT" ]];" we sure will set SPARK_MASTER_IP explicitly, the SPARK_MASTER_PORT option, however, we probably do not set just using spark default port 7077. So if we do not set SPARK_MASTER_PORT, the condition will never be true. We should just use default port if users do not set port explicitly I think.
-rwxr-xr-x | bin/spark-shell | 11 |
1 files changed, 8 insertions, 3 deletions
diff --git a/bin/spark-shell b/bin/spark-shell index e6885b51ef..05a46ee0ca 100755 --- a/bin/spark-shell +++ b/bin/spark-shell @@ -45,13 +45,18 @@ for o in "$@"; do done # Set MASTER from spark-env if possible +DEFAULT_SPARK_MASTER_PORT=7077 if [ -z "$MASTER" ]; then if [ -e "$FWDIR/conf/spark-env.sh" ]; then . "$FWDIR/conf/spark-env.sh" fi - if [[ "x" != "x$SPARK_MASTER_IP" && "y" != "y$SPARK_MASTER_PORT" ]]; then - MASTER="spark://${SPARK_MASTER_IP}:${SPARK_MASTER_PORT}" - export MASTER + if [ "x" != "x$SPARK_MASTER_IP" ]; then + if [ "y" != "y$SPARK_MASTER_PORT" ]; then + SPARK_MASTER_PORT="${SPARK_MASTER_PORT}" + else + SPARK_MASTER_PORT=$DEFAULT_SPARK_MASTER_PORT + fi + export MASTER="spark://${SPARK_MASTER_IP}:${SPARK_MASTER_PORT}" fi fi |