From 8ba2b7f28fee39c4839e5ea125bd25f5091a3a1e Mon Sep 17 00:00:00 2001 From: jerryshao Date: Fri, 1 Apr 2016 10:52:13 -0700 Subject: [SPARK-12343][YARN] Simplify Yarn client and client argument ## What changes were proposed in this pull request? Currently in Spark on YARN, configurations can be passed through SparkConf, env and command arguments, some parts are duplicated, like client argument and SparkConf. So here propose to simplify the command arguments. ## How was this patch tested? This patch is tested manually with unit test. CC vanzin tgravescs , please help to suggest this proposal. The original purpose of this JIRA is to remove `ClientArguments`, through refactoring some arguments like `--class`, `--arg` are not so easy to replace, so here I remove the most part of command line arguments, only keep the minimal set. Author: jerryshao Closes #11603 from jerryshao/SPARK-12343. --- docs/running-on-yarn.md | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'docs/running-on-yarn.md') diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index c775fe710f..bb83272ec8 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -215,6 +215,13 @@ If you need a reference to the proper location to put log files in the YARN so t Comma-separated list of files to be placed in the working directory of each executor. + + spark.yarn.dist.jars + (none) + + Comma-separated list of jars to be placed in the working directory of each executor. + + spark.executor.cores 1 in YARN mode, all the available cores on the worker in standalone mode. -- cgit v1.2.3