aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorStavros Kontopoulos <stavros.kontopoulos@lightbend.com>2016-08-18 12:19:19 +0100
committerSean Owen <sowen@cloudera.com>2016-08-18 12:19:19 +0100
commitb81421afb04959bb22b53653be0a09c1f1c5845f (patch)
treeb627c4379441809ee1a14e0800c209b679e63a42 /docs
parente82dbe600e0d36d76cd5607a77c3243a26777b77 (diff)
downloadspark-b81421afb04959bb22b53653be0a09c1f1c5845f.tar.gz
spark-b81421afb04959bb22b53653be0a09c1f1c5845f.tar.bz2
spark-b81421afb04959bb22b53653be0a09c1f1c5845f.zip
[SPARK-17087][MESOS] Documentation for Making Spark on Mesos honor port restrictions
## What changes were proposed in this pull request? - adds documentation for https://issues.apache.org/jira/browse/SPARK-11714 ## How was this patch tested? Doc no test needed. Author: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com> Closes #14667 from skonto/add_doc.
Diffstat (limited to 'docs')
-rw-r--r--docs/running-on-mesos.md10
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md
index a6ce34c761..173961deaa 100644
--- a/docs/running-on-mesos.md
+++ b/docs/running-on-mesos.md
@@ -207,6 +207,16 @@ The scheduler will start executors round-robin on the offers Mesos
gives it, but there are no spread guarantees, as Mesos does not
provide such guarantees on the offer stream.
+In this mode spark executors will honor port allocation if such is
+provided from the user. Specifically if the user defines
+`spark.executor.port` or `spark.blockManager.port` in Spark configuration,
+the mesos scheduler will check the available offers for a valid port
+range containing the port numbers. If no such range is available it will
+not launch any task. If no restriction is imposed on port numbers by the
+user, ephemeral ports are used as usual. This port honouring implementation
+implies one task per host if the user defines a port. In the future network
+isolation shall be supported.
+
The benefit of coarse-grained mode is much lower startup overhead, but
at the cost of reserving Mesos resources for the complete duration of
the application. To configure your job to dynamically adjust to its