aboutsummaryrefslogtreecommitdiff
path: root/docs/security.md
diff options
context:
space:
mode:
authorMarcelo Vanzin <vanzin@cloudera.com>2015-11-23 13:54:19 -0800
committerMarcelo Vanzin <vanzin@cloudera.com>2015-11-23 13:54:19 -0800
commitc2467dadae8ce44010a912ee91c429310f8add65 (patch)
tree13f1eb5a8e2e2348a6a0b1296a720b8ef626eaf9 /docs/security.md
parent7cfa4c6bc36d97e459d4adee7b03d537d63c337e (diff)
downloadspark-c2467dadae8ce44010a912ee91c429310f8add65.tar.gz
spark-c2467dadae8ce44010a912ee91c429310f8add65.tar.bz2
spark-c2467dadae8ce44010a912ee91c429310f8add65.zip
[SPARK-11140][CORE] Transfer files using network lib when using NettyRpcEnv.
This change abstracts the code that serves jars / files to executors so that each RpcEnv can have its own implementation; the akka version uses the existing HTTP-based file serving mechanism, while the netty versions uses the new stream support added to the network lib, which makes file transfers benefit from the easier security configuration of the network library, and should also reduce overhead overall. The change includes a small fix to TransportChannelHandler so that it propagates user events to downstream handlers. Author: Marcelo Vanzin <vanzin@cloudera.com> Closes #9530 from vanzin/SPARK-11140.
Diffstat (limited to 'docs/security.md')
-rw-r--r--docs/security.md5
1 files changed, 3 insertions, 2 deletions
diff --git a/docs/security.md b/docs/security.md
index 1771094151..e1af221d44 100644
--- a/docs/security.md
+++ b/docs/security.md
@@ -149,7 +149,8 @@ configure those ports.
<td>(random)</td>
<td>Schedule tasks</td>
<td><code>spark.executor.port</code></td>
- <td>Akka-based. Set to "0" to choose a port randomly.</td>
+ <td>Akka-based. Set to "0" to choose a port randomly. Only used if Akka RPC backend is
+ configured.</td>
</tr>
<tr>
<td>Executor</td>
@@ -157,7 +158,7 @@ configure those ports.
<td>(random)</td>
<td>File server for files and jars</td>
<td><code>spark.fileserver.port</code></td>
- <td>Jetty-based</td>
+ <td>Jetty-based. Only used if Akka RPC backend is configured.</td>
</tr>
<tr>
<td>Executor</td>