diff options
| author | Sandy Ryza <sandy@cloudera.com> | 2015-05-11 12:09:39 -0700 |
|---|---|---|
| committer | Sandy Ryza <sandy@cloudera.com> | 2015-05-11 12:09:39 -0700 |
| commit | 82fee9d9aad2c9ba2fb4bd658579fe99218cafac (patch) | |
| tree | ff383391eba6cc7a9038885fade81fe14b0c3182 /docs/running-on-yarn.md | |
| parent | 0a4844f90a712e796c9404b422cea76d21a5d2e3 (diff) | |
| download | spark-82fee9d9aad2c9ba2fb4bd658579fe99218cafac.tar.gz spark-82fee9d9aad2c9ba2fb4bd658579fe99218cafac.tar.bz2 spark-82fee9d9aad2c9ba2fb4bd658579fe99218cafac.zip | |
[SPARK-6470] [YARN] Add support for YARN node labels.
This is difficult to write a test for because it relies on the latest version of YARN, but I verified manually that the patch does pass along the label expression on this version and containers are successfully launched.
Author: Sandy Ryza <sandy@cloudera.com>
Closes #5242 from sryza/sandy-spark-6470 and squashes the following commits:
6af87b9 [Sandy Ryza] Change info to warning
6e22d99 [Sandy Ryza] [YARN] SPARK-6470. Add support for YARN node labels.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 4fb4a90307..51c1339165 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -220,6 +220,15 @@ Most of the configs are the same for Spark on YARN as for other deployment modes Otherwise, the client process will exit after submission. </td> </tr> +<tr> + <td><code>spark.yarn.executor.nodeLabelExpression</code></td> + <td>(none)</td> + <td> + A YARN node label expression that restricts the set of nodes executors will be scheduled on. + Only versions of YARN greater than or equal to 2.6 support node label expressions, so when + running against earlier versions, this property will be ignored. + </td> +</tr> </table> # Launching Spark on YARN |