aboutsummaryrefslogtreecommitdiff
path: root/docs/python-programming-guide.md
Commit message (Collapse)AuthorAgeFilesLines
* SPARK-1183. Don't use "worker" to mean executorSandy Ryza2014-03-131-3/+3
| | | | | | | | | | | | Author: Sandy Ryza <sandy@cloudera.com> Closes #120 from sryza/sandy-spark-1183 and squashes the following commits: 5066a4a [Sandy Ryza] Remove "worker" in a couple comments 0bd1e46 [Sandy Ryza] Remove --am-class from usage bfc8fe0 [Sandy Ryza] Remove am-class from doc and fix yarn-alpha 607539f [Sandy Ryza] Address review comments 74d087a [Sandy Ryza] SPARK-1183. Don't use "worker" to mean executor
* Updated link for pyspark examples in docsJyotiska NK2014-02-261-1/+1
| | | | | | | | Author: Jyotiska NK <jyotiska123@gmail.com> Closes #22 from jyotiska/pyspark_docs and squashes the following commits: 426136c [Jyotiska NK] Updated link for pyspark examples
* Clarify that Python 2.7 is only needed for MLlibMatei Zaharia2014-01-151-2/+2
|
* Update Python required version to 2.7, and mention MLlib supportMatei Zaharia2014-01-121-1/+7
|
* Simplify and fix pyspark script.Patrick Wendell2014-01-071-2/+3
| | | | | | | | | | | | | | | | | | This patch removes compatibility for IPython < 1.0 but fixes the launch script and makes it much simpler. I tested this using the three commands in the PySpark documentation page: 1. IPYTHON=1 ./pyspark 2. IPYTHON_OPTS="notebook" ./pyspark 3. IPYTHON_OPTS="notebook --pylab inline" ./pyspark There are two changes: - We rely on PYTHONSTARTUP env var to start PySpark - Removed the quotes around $IPYTHON_OPTS... having quotes gloms them together as a single argument passed to `exec` which seemed to cause ipython to fail (it instead expects them as multiple arguments).
* Code review feedbackHolden Karau2014-01-051-1/+1
|
* Merge remote-tracking branch 'apache-github/master' into remove-binariesPatrick Wendell2014-01-031-14/+14
|\ | | | | | | | | | | Conflicts: core/src/test/scala/org/apache/spark/DriverSuite.scala docs/python-programming-guide.md
| * pyspark -> bin/pysparkPrashant Sharma2014-01-021-14/+14
| |
* | Merge branch 'master' into spark-1002-remove-jarsPrashant Sharma2014-01-031-7/+8
|\|
| * Updated docs for SparkConf and handled review commentsMatei Zaharia2013-12-301-7/+8
| |
* | Removed sbt folder and changed docs accordinglyPrashant Sharma2014-01-021-1/+1
|/
* Add notes to python documentation about using SparkContext.setSystemProperty.Ewen Cheslack-Postava2013-10-221-0/+11
|
* Fix PySpark docs and an overly long line of code after fdbae41eMatei Zaharia2013-10-091-1/+1
|
* Update Python API featuresMatei Zaharia2013-09-101-1/+1
|
* More fair scheduler docs and property names.Matei Zaharia2013-09-081-6/+6
| | | | | Also changed uses of "job" terminology to "application" when they referred to an entire Spark program, to avoid confusion.
* Doc improvementsMatei Zaharia2013-09-011-18/+18
|
* Fix more URLs in docsMatei Zaharia2013-09-011-2/+5
|
* More updates, describing changes to recommended use of environment varsMatei Zaharia2013-08-311-2/+2
| | | | and new Python stuff
* Update some build instructions because only sbt assembly and mvn packageMatei Zaharia2013-08-291-1/+1
| | | | are now needed
* Add docs about ipythonMatei Zaharia2013-07-291-3/+31
|
* Clarify that PySpark is not supported on Windowsroot2013-07-011-3/+2
|
* Simplify Python docs a little to do substring searchMatei Zaharia2013-06-261-4/+3
|
* Some tweaks to docsMatei Zaharia2013-02-251-2/+2
|
* Added checkpointing and fault-tolerance semantics to the programming guide. ↵Tathagata Das2013-02-181-1/+1
| | | | Fixed default checkpoint interval to being a multiple of slide duration. Fixed visibility of some classes and objects to clean up docs.
* Make module help available in python shell.Patrick Wendell2013-01-301-0/+1
| | | | Also, adds a line in doc explaining how to use.
* Inclue packaging and launching pyspark in guide.Patrick Wendell2013-01-301-2/+8
| | | | It's nicer if all the commands you need are made explicit.
* Fix Python guide to say accumulators are availableMatei Zaharia2013-01-201-1/+0
|
* Add mapPartitionsWithSplit() to PySpark.Josh Rosen2013-01-081-1/+0
|
* Add `pyspark` script to replace the other scripts.Josh Rosen2013-01-011-5/+44
| | | Expand the PySpark programming guide.
* Minor documentation and style fixes for PySpark.Josh Rosen2013-01-011-2/+1
|
* Add documentation for Python API.Josh Rosen2012-12-281-0/+74