From eca58755fbbc11937b335ad953a3caff89b818e6 Mon Sep 17 00:00:00 2001 From: Timothy Chen Date: Wed, 10 Aug 2016 10:11:03 +0100 Subject: [SPARK-16927][SPARK-16923] Override task properties at dispatcher. ## What changes were proposed in this pull request? - enable setting default properties for all jobs submitted through the dispatcher [SPARK-16927] - remove duplication of conf vars on cluster submitted jobs [SPARK-16923] (this is a small fix, so I'm including in the same PR) ## How was this patch tested? mesos/spark integration test suite manual testing Author: Timothy Chen Closes #14511 from mgummelt/override-props. --- docs/running-on-mesos.md | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'docs') diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index 613da68531..a6ce34c761 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -467,6 +467,17 @@ See the [configuration page](configuration.html) for information on Spark config Set the Spark Mesos dispatcher webui_url for interacting with the framework. If unset it will point to Spark's internal web UI. + + + spark.mesos.dispatcher.driverDefault.[PropertyName] + (none) + + Set default properties for drivers submitted through the + dispatcher. For example, + spark.mesos.dispatcher.driverProperty.spark.executor.memory=32g + results in the executors for all drivers submitted in cluster mode + to run in 32g containers. + spark.mesos.dispatcher.historyServer.url -- cgit v1.2.3