diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2015-02-10 17:19:10 -0800 |
|---|---|---|
| committer | Patrick Wendell <patrick@databricks.com> | 2015-02-10 17:19:10 -0800 |
| commit | ed167e70c6d355f39b366ea0d3b92dd26d826a0b (patch) | |
| tree | 08d8fac6279a161d9d83d5aea334215094c2c2c7 /bin | |
| parent | e28b6bdbb5c5e4fd62ec0b547b77719c3f7e476e (diff) | |
| download | spark-ed167e70c6d355f39b366ea0d3b92dd26d826a0b.tar.gz spark-ed167e70c6d355f39b366ea0d3b92dd26d826a0b.tar.bz2 spark-ed167e70c6d355f39b366ea0d3b92dd26d826a0b.zip | |
[SPARK-5493] [core] Add option to impersonate user.
Hadoop has a feature that allows users to impersonate other users
when submitting applications or talking to HDFS, for example. These
impersonated users are referred generally as "proxy users".
Services such as Oozie or Hive use this feature to run applications
as the requesting user.
This change makes SparkSubmit accept a new command line option to
run the application as a proxy user. It also fixes the plumbing
of the user name through the UI (and a couple of other places) to
refer to the correct user running the application, which can be
different than `sys.props("user.name")` even without proxies (e.g.
when using kerberos).
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #4405 from vanzin/SPARK-5493 and squashes the following commits:
df82427 [Marcelo Vanzin] Clarify the reason for the special exception handling.
05bfc08 [Marcelo Vanzin] Remove unneeded annotation.
4840de9 [Marcelo Vanzin] Review feedback.
8af06ff [Marcelo Vanzin] Fix usage string.
2e4fa8f [Marcelo Vanzin] Merge branch 'master' into SPARK-5493
b6c947d [Marcelo Vanzin] Merge branch 'master' into SPARK-5493
0540d38 [Marcelo Vanzin] [SPARK-5493] [core] Add option to impersonate user.
Diffstat (limited to 'bin')
| -rwxr-xr-x | bin/utils.sh | 3 | ||||
| -rw-r--r-- | bin/windows-utils.cmd | 1 |
2 files changed, 3 insertions, 1 deletions
diff --git a/bin/utils.sh b/bin/utils.sh index 2241200082..748dbe345a 100755 --- a/bin/utils.sh +++ b/bin/utils.sh @@ -35,7 +35,8 @@ function gatherSparkSubmitOpts() { --master | --deploy-mode | --class | --name | --jars | --packages | --py-files | --files | \ --conf | --repositories | --properties-file | --driver-memory | --driver-java-options | \ --driver-library-path | --driver-class-path | --executor-memory | --driver-cores | \ - --total-executor-cores | --executor-cores | --queue | --num-executors | --archives) + --total-executor-cores | --executor-cores | --queue | --num-executors | --archives | \ + --proxy-user) if [[ $# -lt 2 ]]; then "$SUBMIT_USAGE_FUNCTION" exit 1; diff --git a/bin/windows-utils.cmd b/bin/windows-utils.cmd index 567b8733f7..0cf9e87ca5 100644 --- a/bin/windows-utils.cmd +++ b/bin/windows-utils.cmd @@ -33,6 +33,7 @@ SET opts="%opts:~1,-1% \<--conf\> \<--properties-file\> \<--driver-memory\> \<-- SET opts="%opts:~1,-1% \<--driver-library-path\> \<--driver-class-path\> \<--executor-memory\>" SET opts="%opts:~1,-1% \<--driver-cores\> \<--total-executor-cores\> \<--executor-cores\> \<--queue\>" SET opts="%opts:~1,-1% \<--num-executors\> \<--archives\> \<--packages\> \<--repositories\>" +SET opts="%opts:~1,-1% \<--proxy-user\>" echo %1 | findstr %opts% >nul if %ERRORLEVEL% equ 0 ( |