diff options
author | Patrick Wendell <pwendell@gmail.com> | 2014-01-09 20:29:51 -0800 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-01-09 20:29:51 -0800 |
commit | 300eaa994c399a0c991c1e39b4dd864a7aa4bdc6 (patch) | |
tree | bbc423ab189d43a809dd7b082cfc8fb00a60e2fe | |
parent | 4b074fac054848ebd3397a3cce0a3e7871d3860c (diff) | |
parent | 77ca9e1ba845c8cbb1566f803b591f6a826b0f1d (diff) | |
download | spark-300eaa994c399a0c991c1e39b4dd864a7aa4bdc6.tar.gz spark-300eaa994c399a0c991c1e39b4dd864a7aa4bdc6.tar.bz2 spark-300eaa994c399a0c991c1e39b4dd864a7aa4bdc6.zip |
Merge pull request #353 from pwendell/ipython-simplify
Simplify and fix pyspark script.
This patch removes compatibility for IPython < 1.0 but fixes the launch
script and makes it much simpler.
I tested this using the three commands in the PySpark documentation page:
1. IPYTHON=1 ./pyspark
2. IPYTHON_OPTS="notebook" ./pyspark
3. IPYTHON_OPTS="notebook --pylab inline" ./pyspark
There are two changes:
- We rely on PYTHONSTARTUP env var to start PySpark
- Removed the quotes around $IPYTHON_OPTS... having quotes
gloms them together as a single argument passed to `exec` which
seemed to cause ipython to fail (it instead expects them as
multiple arguments).
-rwxr-xr-x | bin/pyspark | 7 | ||||
-rw-r--r-- | docs/python-programming-guide.md | 5 |
2 files changed, 4 insertions, 8 deletions
diff --git a/bin/pyspark b/bin/pyspark index d6810f4686..ed6f8da730 100755 --- a/bin/pyspark +++ b/bin/pyspark @@ -59,12 +59,7 @@ if [ -n "$IPYTHON_OPTS" ]; then fi if [[ "$IPYTHON" = "1" ]] ; then - # IPython <1.0.0 doesn't honor PYTHONSTARTUP, while 1.0.0+ does. - # Hence we clear PYTHONSTARTUP and use the -c "%run $IPYTHONSTARTUP" command which works on all versions - # We also force interactive mode with "-i" - IPYTHONSTARTUP=$PYTHONSTARTUP - PYTHONSTARTUP= - exec ipython "$IPYTHON_OPTS" -i -c "%run $IPYTHONSTARTUP" + exec ipython $IPYTHON_OPTS else exec "$PYSPARK_PYTHON" "$@" fi diff --git a/docs/python-programming-guide.md b/docs/python-programming-guide.md index dc187b3efe..c4236f8312 100644 --- a/docs/python-programming-guide.md +++ b/docs/python-programming-guide.md @@ -99,8 +99,9 @@ $ MASTER=local[4] ./bin/pyspark ## IPython -It is also possible to launch PySpark in [IPython](http://ipython.org), the enhanced Python interpreter. -To do this, set the `IPYTHON` variable to `1` when running `bin/pyspark`: +It is also possible to launch PySpark in [IPython](http://ipython.org), the +enhanced Python interpreter. PySpark works with IPython 1.0.0 and later. To +use IPython, set the `IPYTHON` variable to `1` when running `bin/pyspark`: {% highlight bash %} $ IPYTHON=1 ./bin/pyspark |