aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMatei Zaharia <matei@eecs.berkeley.edu>2013-01-30 15:35:29 -0800
committerMatei Zaharia <matei@eecs.berkeley.edu>2013-01-30 15:35:29 -0800
commit55327a283e962652a126d3f8ac7e9a19c76f1f19 (patch)
treeb949906a83a8e47f9340292eb1bf579459bb113e
parentd12330bd2cd354919c414ee7ef367bc28c0eb7c9 (diff)
parent3f945e3b830c5a7d50acd61c5aabf964f40f7f4b (diff)
downloadspark-55327a283e962652a126d3f8ac7e9a19c76f1f19.tar.gz
spark-55327a283e962652a126d3f8ac7e9a19c76f1f19.tar.bz2
spark-55327a283e962652a126d3f8ac7e9a19c76f1f19.zip
Merge pull request #430 from pwendell/pyspark-guide
Minor improvements to PySpark docs
-rw-r--r--docs/python-programming-guide.md11
-rw-r--r--python/pyspark/shell.py1
2 files changed, 10 insertions, 2 deletions
diff --git a/docs/python-programming-guide.md b/docs/python-programming-guide.md
index a840b9b34b..4e84d23edf 100644
--- a/docs/python-programming-guide.md
+++ b/docs/python-programming-guide.md
@@ -67,13 +67,20 @@ The script automatically adds the `pyspark` package to the `PYTHONPATH`.
# Interactive Use
-The `pyspark` script launches a Python interpreter that is configured to run PySpark jobs.
-When run without any input files, `pyspark` launches a shell that can be used explore data interactively, which is a simple way to learn the API:
+The `pyspark` script launches a Python interpreter that is configured to run PySpark jobs. To use `pyspark` interactively, first build Spark, then launch it directly from the command line without any options:
+
+{% highlight bash %}
+$ sbt/sbt package
+$ ./pyspark
+{% endhighlight %}
+
+The Python shell can be used explore data interactively and is a simple way to learn the API:
{% highlight python %}
>>> words = sc.textFile("/usr/share/dict/words")
>>> words.filter(lambda w: w.startswith("spar")).take(5)
[u'spar', u'sparable', u'sparada', u'sparadrap', u'sparagrass']
+>>> help(pyspark) # Show all pyspark functions
{% endhighlight %}
By default, the `pyspark` shell creates SparkContext that runs jobs locally.
diff --git a/python/pyspark/shell.py b/python/pyspark/shell.py
index f6328c561f..54ff1bf8e7 100644
--- a/python/pyspark/shell.py
+++ b/python/pyspark/shell.py
@@ -4,6 +4,7 @@ An interactive shell.
This file is designed to be launched as a PYTHONSTARTUP script.
"""
import os
+import pyspark
from pyspark.context import SparkContext