diff options
author | Patrick Wendell <pwendell@gmail.com> | 2014-01-07 17:46:02 -0800 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-01-07 17:55:25 -0800 |
commit | 82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a (patch) | |
tree | af9d4dc940da7bbcb3113c31df443dba1f4ddb0a /bin | |
parent | 6ccf8ce705f13b5c6993675ee034db64363caa31 (diff) | |
download | spark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.tar.gz spark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.tar.bz2 spark-82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a.zip |
Simplify and fix pyspark script.
This patch removes compatibility for IPython < 1.0 but fixes the launch
script and makes it much simpler.
I tested this using the three commands in the PySpark documentation page:
1. IPYTHON=1 ./pyspark
2. IPYTHON_OPTS="notebook" ./pyspark
3. IPYTHON_OPTS="notebook --pylab inline" ./pyspark
There are two changes:
- We rely on PYTHONSTARTUP env var to start PySpark
- Removed the quotes around $IPYTHON_OPTS... having quotes
gloms them together as a single argument passed to `exec` which
seemed to cause ipython to fail (it instead expects them as
multiple arguments).
Diffstat (limited to 'bin')
-rwxr-xr-x | bin/pyspark | 8 |
1 files changed, 1 insertions, 7 deletions
diff --git a/bin/pyspark b/bin/pyspark index d6810f4686..a70da5674e 100755 --- a/bin/pyspark +++ b/bin/pyspark @@ -51,7 +51,6 @@ export PYSPARK_PYTHON export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH # Load the PySpark shell.py script when ./pyspark is used interactively: -export OLD_PYTHONSTARTUP=$PYTHONSTARTUP export PYTHONSTARTUP=$FWDIR/python/pyspark/shell.py if [ -n "$IPYTHON_OPTS" ]; then @@ -59,12 +58,7 @@ if [ -n "$IPYTHON_OPTS" ]; then fi if [[ "$IPYTHON" = "1" ]] ; then - # IPython <1.0.0 doesn't honor PYTHONSTARTUP, while 1.0.0+ does. - # Hence we clear PYTHONSTARTUP and use the -c "%run $IPYTHONSTARTUP" command which works on all versions - # We also force interactive mode with "-i" - IPYTHONSTARTUP=$PYTHONSTARTUP - PYTHONSTARTUP= - exec ipython "$IPYTHON_OPTS" -i -c "%run $IPYTHONSTARTUP" + exec ipython $IPYTHON_OPTS else exec "$PYSPARK_PYTHON" "$@" fi |