diff options
author | Matei Zaharia <matei@databricks.com> | 2013-12-30 22:17:28 -0500 |
---|---|---|
committer | Matei Zaharia <matei@databricks.com> | 2013-12-30 22:17:28 -0500 |
commit | 0fa5809768cf60ec62b4277f04e23a44dc1582e2 (patch) | |
tree | fee16620755769a70975c41d894db43633b18098 /docs/python-programming-guide.md | |
parent | 994f080f8ae3372366e6004600ba791c8a372ff0 (diff) | |
download | spark-0fa5809768cf60ec62b4277f04e23a44dc1582e2.tar.gz spark-0fa5809768cf60ec62b4277f04e23a44dc1582e2.tar.bz2 spark-0fa5809768cf60ec62b4277f04e23a44dc1582e2.zip |
Updated docs for SparkConf and handled review comments
Diffstat (limited to 'docs/python-programming-guide.md')
-rw-r--r-- | docs/python-programming-guide.md | 15 |
1 files changed, 8 insertions, 7 deletions
diff --git a/docs/python-programming-guide.md b/docs/python-programming-guide.md index 55e39b1de1..96f93e24fe 100644 --- a/docs/python-programming-guide.md +++ b/docs/python-programming-guide.md @@ -131,15 +131,16 @@ sc = SparkContext("local", "App Name", pyFiles=['MyFile.py', 'lib.zip', 'app.egg Files listed here will be added to the `PYTHONPATH` and shipped to remote worker machines. Code dependencies can be added to an existing SparkContext using its `addPyFile()` method. -You can set [system properties](configuration.html#system-properties) -using `SparkContext.setSystemProperty()` class method *before* -instantiating SparkContext. For example, to set the amount of memory -per executor process: +You can set [configuration properties](configuration.html#spark-properties) by passing a +[SparkConf](api/pyspark/pyspark.conf.SparkConf-class.html) object to SparkContext: {% highlight python %} -from pyspark import SparkContext -SparkContext.setSystemProperty('spark.executor.memory', '2g') -sc = SparkContext("local", "App Name") +from pyspark import SparkConf, SparkContext +conf = (SparkConf() + .setMaster("local") + .setAppName("My app") + .set("spark.executor.memory", "1g")) +sc = SparkContext(conf = conf) {% endhighlight %} # API Docs |