diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2015-06-05 14:32:00 +0200 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-06-05 14:32:00 +0200 |
| commit | 700312e12f9588f01a592d6eac7bff7eb366ac8f (patch) | |
| tree | d0429a8bb9e78dd2970c12e332a2610f6898d313 /bin/spark-submit | |
| parent | 019dc9f558cf7c0b708d3b1f0882b0c19134ffb6 (diff) | |
| download | spark-700312e12f9588f01a592d6eac7bff7eb366ac8f.tar.gz spark-700312e12f9588f01a592d6eac7bff7eb366ac8f.tar.bz2 spark-700312e12f9588f01a592d6eac7bff7eb366ac8f.zip | |
[SPARK-6324] [CORE] Centralize handling of script usage messages.
Reorganize code so that the launcher library handles most of the work
of printing usage messages, instead of having an awkward protocol between
the library and the scripts for that.
This mostly applies to SparkSubmit, since the launcher lib does not do
command line parsing for classes invoked in other ways, and thus cannot
handle failures for those. Most scripts end up going through SparkSubmit,
though, so it all works.
The change adds a new, internal command line switch, "--usage-error",
which prints the usage message and exits with a non-zero status. Scripts
can override the command printed in the usage message by setting an
environment variable - this avoids having to grep the output of
SparkSubmit to remove references to the "spark-submit" script.
The only sub-optimal part of the change is the special handling for the
spark-sql usage, which is now done in SparkSubmitArguments.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #5841 from vanzin/SPARK-6324 and squashes the following commits:
2821481 [Marcelo Vanzin] Merge branch 'master' into SPARK-6324
bf139b5 [Marcelo Vanzin] Filter output of Spark SQL CLI help.
c6609bf [Marcelo Vanzin] Fix exit code never being used when printing usage messages.
6bc1b41 [Marcelo Vanzin] [SPARK-6324] [core] Centralize handling of script usage messages.
Diffstat (limited to 'bin/spark-submit')
| -rwxr-xr-x | bin/spark-submit | 12 |
1 files changed, 0 insertions, 12 deletions
diff --git a/bin/spark-submit b/bin/spark-submit index 0e0afe71a0..255378b0f0 100755 --- a/bin/spark-submit +++ b/bin/spark-submit @@ -22,16 +22,4 @@ SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)" # disable randomized hash for string in Python 3.3+ export PYTHONHASHSEED=0 -# Only define a usage function if an upstream script hasn't done so. -if ! type -t usage >/dev/null 2>&1; then - usage() { - if [ -n "$1" ]; then - echo "$1" - fi - "$SPARK_HOME"/bin/spark-class org.apache.spark.deploy.SparkSubmit --help - exit "$2" - } - export -f usage -fi - exec "$SPARK_HOME"/bin/spark-class org.apache.spark.deploy.SparkSubmit "$@" |