aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCharles Allen <charles@allen-net.com>2016-11-01 13:14:17 +0000
committerSean Owen <sowen@cloudera.com>2016-11-01 13:14:17 +0000
commite34b4e12673fb76c92f661d7c03527410857a0f8 (patch)
treeb9accb74292f7f7f2e5b9dbc1c5201a9b103a9df /docs
parentcb80edc26349e2e358d27fe2ae8e5d6959b77fab (diff)
downloadspark-e34b4e12673fb76c92f661d7c03527410857a0f8.tar.gz
spark-e34b4e12673fb76c92f661d7c03527410857a0f8.tar.bz2
spark-e34b4e12673fb76c92f661d7c03527410857a0f8.zip
[SPARK-15994][MESOS] Allow enabling Mesos fetch cache in coarse executor backend
Mesos 0.23.0 introduces a Fetch Cache feature http://mesos.apache.org/documentation/latest/fetcher/ which allows caching of resources specified in command URIs. This patch: - Updates the Mesos shaded protobuf dependency to 0.23.0 - Allows setting `spark.mesos.fetcherCache.enable` to enable the fetch cache for all specified URIs. (URIs must be specified for the setting to have any affect) - Updates documentation for Mesos configuration with the new setting. This patch does NOT: - Allow for per-URI caching configuration. The cache setting is global to ALL URIs for the command. Author: Charles Allen <charles@allen-net.com> Closes #13713 from drcrallen/SPARK15994.
Diffstat (limited to 'docs')
-rw-r--r--docs/running-on-mesos.md9
1 files changed, 7 insertions, 2 deletions
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md
index 77b06fcf33..923d8dbebf 100644
--- a/docs/running-on-mesos.md
+++ b/docs/running-on-mesos.md
@@ -506,8 +506,13 @@ See the [configuration page](configuration.html) for information on Spark config
since this configuration is just a upper limit and not a guaranteed amount.
</td>
</tr>
-
-
+<tr>
+ <td><code>spark.mesos.fetcherCache.enable</code></td>
+ <td><code>false</code></td>
+ <td>
+ If set to `true`, all URIs (example: `spark.executor.uri`, `spark.mesos.uris`) will be cached by the [Mesos fetcher cache](http://mesos.apache.org/documentation/latest/fetcher/)
+ </td>
+</tr>
</table>
# Troubleshooting and Debugging