diff options
| author | Kousuke Saruta <sarutak@oss.nttdata.co.jp> | 2014-08-09 21:10:43 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-08-09 21:11:00 -0700 |
| commit | 4f4a9884d9268ba9808744b3d612ac23c75f105a (patch) | |
| tree | 8cf35287ff81866efd3b3ce95d45ce08891b21a0 /core | |
| parent | e45daf226d780f4a7aaabc2de9f04367bee16f26 (diff) | |
| download | spark-4f4a9884d9268ba9808744b3d612ac23c75f105a.tar.gz spark-4f4a9884d9268ba9808744b3d612ac23c75f105a.tar.bz2 spark-4f4a9884d9268ba9808744b3d612ac23c75f105a.zip | |
[SPARK-2894] spark-shell doesn't accept flags
As sryza reported, spark-shell doesn't accept any flags.
The root cause is wrong usage of spark-submit in spark-shell and it come to the surface by #1801
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Author: Cheng Lian <lian.cs.zju@gmail.com>
Closes #1715, Closes #1864, and Closes #1861
Closes #1825 from sarutak/SPARK-2894 and squashes the following commits:
47f3510 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2894
2c899ed [Kousuke Saruta] Removed useless code from java_gateway.py
98287ed [Kousuke Saruta] Removed useless code from java_gateway.py
513ad2e [Kousuke Saruta] Modified util.sh to enable to use option including white spaces
28a374e [Kousuke Saruta] Modified java_gateway.py to recognize arguments
5afc584 [Cheng Lian] Filter out spark-submit options when starting Python gateway
e630d19 [Cheng Lian] Fixing pyspark and spark-shell CLI options
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala index c21f1529a1..d545f58c5d 100644 --- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala +++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala @@ -224,6 +224,10 @@ private[spark] class SparkSubmitArguments(args: Seq[String]) { // Delineates parsing of Spark options from parsing of user options. parse(opts) + /** + * NOTE: If you add or remove spark-submit options, + * modify NOT ONLY this file but also utils.sh + */ def parse(opts: Seq[String]): Unit = opts match { case ("--name") :: value :: tail => name = value |