diff options
| author | Michael Gummelt <mgummelt@mesosphere.io> | 2016-07-29 05:50:47 -0700 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-07-29 05:50:47 -0700 |
| commit | 266b92faffb66af24d8ed2725beb80770a2d91f8 (patch) | |
| tree | 408d3ccb12dcb41ae577600eb1fadaebe79f5482 /docs | |
| parent | 04a2c072d94874f3f7ae9dd94c026e8826a75ccd (diff) | |
| download | spark-266b92faffb66af24d8ed2725beb80770a2d91f8.tar.gz spark-266b92faffb66af24d8ed2725beb80770a2d91f8.tar.bz2 spark-266b92faffb66af24d8ed2725beb80770a2d91f8.zip | |
[SPARK-16637] Unified containerizer
## What changes were proposed in this pull request?
New config var: spark.mesos.docker.containerizer={"mesos","docker" (default)}
This adds support for running docker containers via the Mesos unified containerizer: http://mesos.apache.org/documentation/latest/container-image/
The benefit is losing the dependency on `dockerd`, and all the costs which it incurs.
I've also updated the supported Mesos version to 0.28.2 for support of the required protobufs.
This is blocked on: https://github.com/apache/spark/pull/14167
## How was this patch tested?
- manually testing jobs submitted with both "mesos" and "docker" settings for the new config var.
- spark/mesos integration test suite
Author: Michael Gummelt <mgummelt@mesosphere.io>
Closes #14275 from mgummelt/unified-containerizer.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/_config.yml | 2 | ||||
| -rw-r--r-- | docs/running-on-mesos.md | 10 |
2 files changed, 11 insertions, 1 deletions
diff --git a/docs/_config.yml b/docs/_config.yml index bbb576e0e7..e4fc093fe7 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -18,6 +18,6 @@ SPARK_VERSION: 2.1.0-SNAPSHOT SPARK_VERSION_SHORT: 2.1.0 SCALA_BINARY_VERSION: "2.11" SCALA_VERSION: "2.11.7" -MESOS_VERSION: 0.22.0 +MESOS_VERSION: 1.0.0 SPARK_ISSUE_TRACKER_URL: https://issues.apache.org/jira/browse/SPARK SPARK_GITHUB_URL: https://github.com/apache/spark diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index ce888b5445..d037e7be0a 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -433,6 +433,16 @@ See the [configuration page](configuration.html) for information on Spark config </td> </tr> <tr> + <td><code>spark.mesos.containerizer</code></td> + <td><code>docker</code></td> + <td> + This only affects docker containers, and must be one of "docker" + or "mesos". Mesos supports two types of + containerizers for docker: the "docker" containerizer, and the preferred + "mesos" containerizer. Read more here: http://mesos.apache.org/documentation/latest/container-image/ + </td> +</tr> +<tr> <td><code>spark.mesos.driver.webui.url</code></td> <td><code>(none)</code></td> <td> |