diff options
author | Patrick Wendell <pwendell@gmail.com> | 2014-03-31 12:07:14 -0700 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-03-31 12:07:14 -0700 |
commit | 841721e03cc44ee7d8fe72c882db8c0f9f3af365 (patch) | |
tree | 27565f026eb4700822b66471d062c0d586f1d2a6 /yarn/stable/src | |
parent | d666053679ded5a32088c1758d20101126e23af6 (diff) | |
download | spark-841721e03cc44ee7d8fe72c882db8c0f9f3af365.tar.gz spark-841721e03cc44ee7d8fe72c882db8c0f9f3af365.tar.bz2 spark-841721e03cc44ee7d8fe72c882db8c0f9f3af365.zip |
SPARK-1352: Improve robustness of spark-submit script
1. Better error messages when required arguments are missing.
2. Support for unit testing cases where presented arguments are invalid.
3. Bug fix: Only use environment varaibles when they are set (otherwise will cause NPE).
4. A verbose mode to aid debugging.
5. Visibility of several variables is set to private.
6. Deprecation warning for existing scripts.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #271 from pwendell/spark-submit and squashes the following commits:
9146def [Patrick Wendell] SPARK-1352: Improve robustness of spark-submit script
Diffstat (limited to 'yarn/stable/src')
-rw-r--r-- | yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala index 837b7e12cb..77eb1276a0 100644 --- a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala +++ b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala @@ -173,6 +173,9 @@ class Client(clientArgs: ClientArguments, hadoopConf: Configuration, spConf: Spa object Client { def main(argStrings: Array[String]) { + println("WARNING: This client is deprecated and will be removed in a future version of Spark.") + println("Use ./bin/spark-submit with \"--master yarn\"") + // Set an env variable indicating we are running in YARN mode. // Note: anything env variable with SPARK_ prefix gets propagated to all (remote) processes - // see Client#setupLaunchEnv(). |