diff options
| author | WangTaoTheTonic <wangtao111@huawei.com> | 2015-05-29 11:06:11 -0500 |
|---|---|---|
| committer | Thomas Graves <tgraves@thatenemy-lm.champ.corp.yahoo.com> | 2015-05-29 11:06:11 -0500 |
| commit | a51b133de3c65a991ab105b6f020082080121b4c (patch) | |
| tree | 3b61410c464fdc41f365c6cd5305bf39e61dfe7d /docs/running-on-yarn.md | |
| parent | 8db40f6711058c3c3bf67ceaaaffffcc25d67d19 (diff) | |
| download | spark-a51b133de3c65a991ab105b6f020082080121b4c.tar.gz spark-a51b133de3c65a991ab105b6f020082080121b4c.tar.bz2 spark-a51b133de3c65a991ab105b6f020082080121b4c.zip | |
[SPARK-7524] [SPARK-7846] add configs for keytab and principal, pass these two configs with different way in different modes
* As spark now supports long running service by updating tokens for namenode, but only accept parameters passed with "--k=v" format which is not very convinient. This patch add spark.* configs in properties file and system property.
* --principal and --keytabl options are passed to client but when we started thrift server or spark-shell these two are also passed into the Main class (org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 and org.apache.spark.repl.Main).
In these two main class, arguments passed in will be processed with some 3rd libraries, which will lead to some error: "Invalid option: --principal" or "Unrecgnised option: --principal".
We should pass these command args in different forms, say system properties.
Author: WangTaoTheTonic <wangtao111@huawei.com>
Closes #6051 from WangTaoTheTonic/SPARK-7524 and squashes the following commits:
e65699a [WangTaoTheTonic] change logic to loadEnvironments
ebd9ea0 [WangTaoTheTonic] merge master
ecfe43a [WangTaoTheTonic] pass keytab and principal seperately in different mode
33a7f40 [WangTaoTheTonic] expand the use of the current configs
08bb4e8 [WangTaoTheTonic] fix wrong cite
73afa64 [WangTaoTheTonic] add configs for keytab and principal, move originals to internal
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 16 |
1 files changed, 16 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 9d55f435e8..96cf612c54 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -242,6 +242,22 @@ Most of the configs are the same for Spark on YARN as for other deployment modes running against earlier versions, this property will be ignored. </td> </tr> +<tr> + <td><code>spark.yarn.keytab</code></td> + <td>(none)</td> + <td> + The full path to the file that contains the keytab for the principal specified above. + This keytab will be copied to the node running the Application Master via the Secure Distributed Cache, + for renewing the login tickets and the delegation tokens periodically. + </td> +</tr> +<tr> + <td><code>spark.yarn.principal</code></td> + <td>(none)</td> + <td> + Principal to be used to login to KDC, while running on secure HDFS. + </td> +</tr> </table> # Launching Spark on YARN |