diff options
| author | Michael Gummelt <mgummelt@mesosphere.io> | 2016-07-21 18:29:00 +0100 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-07-21 18:29:00 +0100 |
| commit | 235cb256d06653bcde4c3ed6b081503a94996321 (patch) | |
| tree | 3c1f51a312cc92e120aad31607ddbb438aff5d1f /docs | |
| parent | 69626adddc0441a4834b70a32e2d95b11d69a219 (diff) | |
| download | spark-235cb256d06653bcde4c3ed6b081503a94996321.tar.gz spark-235cb256d06653bcde4c3ed6b081503a94996321.tar.bz2 spark-235cb256d06653bcde4c3ed6b081503a94996321.zip | |
[SPARK-16194] Mesos Driver env vars
## What changes were proposed in this pull request?
Added new configuration namespace: spark.mesos.env.*
This allows a user submitting a job in cluster mode to set arbitrary environment variables on the driver.
spark.mesos.driverEnv.KEY=VAL will result in the env var "KEY" being set to "VAL"
I've also refactored the tests a bit so we can re-use code in MesosClusterScheduler.
And I've refactored the command building logic in `buildDriverCommand`. Command builder values were very intertwined before, and now it's easier to determine exactly how each variable is set.
## How was this patch tested?
unit tests
Author: Michael Gummelt <mgummelt@mesosphere.io>
Closes #14167 from mgummelt/driver-env-vars.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/running-on-mesos.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index 5219e99fee..10dc9ce890 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -429,6 +429,16 @@ See the [configuration page](configuration.html) for information on Spark config </td> </tr> <tr> + <td><code>spark.mesos.driverEnv.[EnvironmentVariableName]</code></td> + <td><code>(none)</code></td> + <td> + This only affects drivers submitted in cluster mode. Add the + environment variable specified by EnvironmentVariableName to the + driver process. The user can specify multiple of these to set + multiple environment variables. + </td> +</tr> +<tr> <td><code>spark.mesos.dispatcher.webui.url</code></td> <td><code>(none)</code></td> <td> |