aboutsummaryrefslogtreecommitdiff
path: root/docs/programming-guide.md
diff options
context:
space:
mode:
authorDavies Liu <davies@databricks.com>2015-08-18 22:11:27 -0700
committerReynold Xin <rxin@databricks.com>2015-08-18 22:11:27 -0700
commitde3223872a217c5224ba7136604f6b7753b29108 (patch)
treefe53feda0f368e08d57db7386cc10dca871822b0 /docs/programming-guide.md
parent1ff0580eda90f9247a5233809667f5cebaea290e (diff)
downloadspark-de3223872a217c5224ba7136604f6b7753b29108.tar.gz
spark-de3223872a217c5224ba7136604f6b7753b29108.tar.bz2
spark-de3223872a217c5224ba7136604f6b7753b29108.zip
[SPARK-9705] [DOC] fix docs about Python version
cc JoshRosen Author: Davies Liu <davies@databricks.com> Closes #8245 from davies/python_doc.
Diffstat (limited to 'docs/programming-guide.md')
-rw-r--r--docs/programming-guide.md12
1 files changed, 10 insertions, 2 deletions
diff --git a/docs/programming-guide.md b/docs/programming-guide.md
index ae712d6274..982c5eabe6 100644
--- a/docs/programming-guide.md
+++ b/docs/programming-guide.md
@@ -85,8 +85,8 @@ import org.apache.spark.SparkConf
<div data-lang="python" markdown="1">
-Spark {{site.SPARK_VERSION}} works with Python 2.6 or higher (but not Python 3). It uses the standard CPython interpreter,
-so C libraries like NumPy can be used.
+Spark {{site.SPARK_VERSION}} works with Python 2.6+ or Python 3.4+. It can use the standard CPython interpreter,
+so C libraries like NumPy can be used. It also works with PyPy 2.3+.
To run Spark applications in Python, use the `bin/spark-submit` script located in the Spark directory.
This script will load Spark's Java/Scala libraries and allow you to submit applications to a cluster.
@@ -104,6 +104,14 @@ Finally, you need to import some Spark classes into your program. Add the follow
from pyspark import SparkContext, SparkConf
{% endhighlight %}
+PySpark requires the same minor version of Python in both driver and workers. It uses the default python version in PATH,
+you can specify which version of Python you want to use by `PYSPARK_PYTHON`, for example:
+
+{% highlight bash %}
+$ PYSPARK_PYTHON=python3.4 bin/pyspark
+$ PYSPARK_PYTHON=/opt/pypy-2.5/bin/pypy bin/spark-submit examples/src/main/python/pi.py
+{% endhighlight %}
+
</div>
</div>