diff options
| author | Andrew Ash <andrew@andrewash.com> | 2014-01-21 14:49:35 -0800 |
|---|---|---|
| committer | Andrew Ash <andrew@andrewash.com> | 2014-01-21 14:49:35 -0800 |
| commit | 069bb94206530a30d095c4920ecfc3b4a8635a72 (patch) | |
| tree | 8c2d5441a6bdd72543d9181239f0daacb2b5c4e3 /docs/configuration.md | |
| parent | f8544981a6d05687fa950639cb1eb3c31e9b6bf5 (diff) | |
| download | spark-069bb94206530a30d095c4920ecfc3b4a8635a72.tar.gz spark-069bb94206530a30d095c4920ecfc3b4a8635a72.tar.bz2 spark-069bb94206530a30d095c4920ecfc3b4a8635a72.zip | |
Clarify spark.default.parallelism
It's the task count across the cluster, not per worker, per machine, per core, or anything else.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index be548e372d..3bb655075f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -98,7 +98,7 @@ Apart from these, the following properties are also available, and may be useful <td>spark.default.parallelism</td> <td>8</td> <td> - Default number of tasks to use for distributed shuffle operations (<code>groupByKey</code>, + Default number of tasks to use across the cluster for distributed shuffle operations (<code>groupByKey</code>, <code>reduceByKey</code>, etc) when not set by user. </td> </tr> |