aboutsummaryrefslogtreecommitdiff
path: root/bin
diff options
context:
space:
mode:
authorPatrick Wendell <pwendell@gmail.com>2014-01-07 17:46:02 -0800
committerPatrick Wendell <pwendell@gmail.com>2014-01-07 17:55:25 -0800
commit82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a (patch)
treeaf9d4dc940da7bbcb3113c31df443dba1f4ddb0a /bin
parent6ccf8ce705f13b5c6993675ee034db64363caa31 (diff)
downloadspark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.tar.gz
spark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.tar.bz2
spark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.zip
Simplify and fix pyspark script.
This patch removes compatibility for IPython < 1.0 but fixes the launch script and makes it much simpler. I tested this using the three commands in the PySpark documentation page: 1. IPYTHON=1 ./pyspark 2. IPYTHON_OPTS="notebook" ./pyspark 3. IPYTHON_OPTS="notebook --pylab inline" ./pyspark There are two changes: - We rely on PYTHONSTARTUP env var to start PySpark - Removed the quotes around $IPYTHON_OPTS... having quotes gloms them together as a single argument passed to `exec` which seemed to cause ipython to fail (it instead expects them as multiple arguments).
Diffstat (limited to 'bin')
-rwxr-xr-xbin/pyspark8
1 files changed, 1 insertions, 7 deletions
diff --git a/bin/pyspark b/bin/pyspark
index d6810f4686..a70da5674e 100755
--- a/bin/pyspark
+++ b/bin/pyspark
@@ -51,7 +51,6 @@ export PYSPARK_PYTHON
export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH
# Load the PySpark shell.py script when ./pyspark is used interactively:
-export OLD_PYTHONSTARTUP=$PYTHONSTARTUP
export PYTHONSTARTUP=$FWDIR/python/pyspark/shell.py
if [ -n "$IPYTHON_OPTS" ]; then
@@ -59,12 +58,7 @@ if [ -n "$IPYTHON_OPTS" ]; then
fi
if [[ "$IPYTHON" = "1" ]] ; then
- # IPython <1.0.0 doesn't honor PYTHONSTARTUP, while 1.0.0+ does.
- # Hence we clear PYTHONSTARTUP and use the -c "%run $IPYTHONSTARTUP" command which works on all versions
- # We also force interactive mode with "-i"
- IPYTHONSTARTUP=$PYTHONSTARTUP
- PYTHONSTARTUP=
- exec ipython "$IPYTHON_OPTS" -i -c "%run $IPYTHONSTARTUP"
+ exec ipython $IPYTHON_OPTS
else
exec "$PYSPARK_PYTHON" "$@"
fi