diff options
author | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-01-30 15:35:29 -0800 |
---|---|---|
committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-01-30 15:35:29 -0800 |
commit | 55327a283e962652a126d3f8ac7e9a19c76f1f19 (patch) | |
tree | b949906a83a8e47f9340292eb1bf579459bb113e /docs | |
parent | d12330bd2cd354919c414ee7ef367bc28c0eb7c9 (diff) | |
parent | 3f945e3b830c5a7d50acd61c5aabf964f40f7f4b (diff) | |
download | spark-55327a283e962652a126d3f8ac7e9a19c76f1f19.tar.gz spark-55327a283e962652a126d3f8ac7e9a19c76f1f19.tar.bz2 spark-55327a283e962652a126d3f8ac7e9a19c76f1f19.zip |
Merge pull request #430 from pwendell/pyspark-guide
Minor improvements to PySpark docs
Diffstat (limited to 'docs')
-rw-r--r-- | docs/python-programming-guide.md | 11 |
1 files changed, 9 insertions, 2 deletions
diff --git a/docs/python-programming-guide.md b/docs/python-programming-guide.md index a840b9b34b..4e84d23edf 100644 --- a/docs/python-programming-guide.md +++ b/docs/python-programming-guide.md @@ -67,13 +67,20 @@ The script automatically adds the `pyspark` package to the `PYTHONPATH`. # Interactive Use -The `pyspark` script launches a Python interpreter that is configured to run PySpark jobs. -When run without any input files, `pyspark` launches a shell that can be used explore data interactively, which is a simple way to learn the API: +The `pyspark` script launches a Python interpreter that is configured to run PySpark jobs. To use `pyspark` interactively, first build Spark, then launch it directly from the command line without any options: + +{% highlight bash %} +$ sbt/sbt package +$ ./pyspark +{% endhighlight %} + +The Python shell can be used explore data interactively and is a simple way to learn the API: {% highlight python %} >>> words = sc.textFile("/usr/share/dict/words") >>> words.filter(lambda w: w.startswith("spar")).take(5) [u'spar', u'sparable', u'sparada', u'sparadrap', u'sparagrass'] +>>> help(pyspark) # Show all pyspark functions {% endhighlight %} By default, the `pyspark` shell creates SparkContext that runs jobs locally. |