aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorSean Owen <sowen@cloudera.com>2015-08-06 19:29:42 +0100
committerSean Owen <sowen@cloudera.com>2015-08-06 19:29:42 +0100
commit0d7aac99da660cc42eb5a9be8e262bd9bd8a770f (patch)
treef5b956d776a2aca9a2c7dd3f28b9e08009293e9b /docs
parentcdd53b762bf358616b313e3334b5f6945caf9ab1 (diff)
downloadspark-0d7aac99da660cc42eb5a9be8e262bd9bd8a770f.tar.gz
spark-0d7aac99da660cc42eb5a9be8e262bd9bd8a770f.tar.bz2
spark-0d7aac99da660cc42eb5a9be8e262bd9bd8a770f.zip
[SPARK-9641] [DOCS] spark.shuffle.service.port is not documented
Document spark.shuffle.service.{enabled,port} CC sryza tgravescs This is pretty minimal; is there more to say here about the service? Author: Sean Owen <sowen@cloudera.com> Closes #7991 from srowen/SPARK-9641 and squashes the following commits: 3bb946e [Sean Owen] Add link to docs for setup and config of external shuffle service 2302e01 [Sean Owen] Document spark.shuffle.service.{enabled,port}
Diffstat (limited to 'docs')
-rw-r--r--docs/configuration.md19
1 files changed, 19 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md
index 24b606356a..c60dd16839 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -474,6 +474,25 @@ Apart from these, the following properties are also available, and may be useful
</td>
</tr>
<tr>
+ <td><code>spark.shuffle.service.enabled</code></td>
+ <td>false</td>
+ <td>
+ Enables the external shuffle service. This service preserves the shuffle files written by
+ executors so the executors can be safely removed. This must be enabled if
+ <code>spark.dynamicAllocation.enabled</code> is "true". The external shuffle service
+ must be set up in order to enable it. See
+ <a href="job-scheduling.html#configuration-and-setup">dynamic allocation
+ configuration and setup documentation</a> for more information.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.shuffle.service.port</code></td>
+ <td>7337</td>
+ <td>
+ Port on which the external shuffle service will run.
+ </td>
+</tr>
+<tr>
<td><code>spark.shuffle.sort.bypassMergeThreshold</code></td>
<td>200</td>
<td>