From 6dd64e8bb2256b56e0908c628ebdb3b533adf432 Mon Sep 17 00:00:00 2001 From: "Y.CORP.YAHOO.COM\\tgraves" Date: Mon, 26 Aug 2013 14:29:24 -0500 Subject: Update docs and remove old reference to --user option --- docs/running-on-yarn.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) (limited to 'docs/running-on-yarn.md') diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 6bada9bdd7..7a344b3ce2 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -57,7 +57,6 @@ The command to launch the YARN Client is as follows: --master-memory \ --worker-memory \ --worker-cores \ - --user \ --queue For example: @@ -77,5 +76,4 @@ The above starts a YARN Client programs which periodically polls the Application - When your application instantiates a Spark context it must use a special "yarn-standalone" master url. This starts the scheduler without forcing it to connect to a cluster. A good way to handle this is to pass "yarn-standalone" as an argument to your program, as shown in the example above. - We do not requesting container resources based on the number of cores. Thus the numbers of cores given via command line arguments cannot be guaranteed. -- Currently, we have not yet integrated with hadoop security. If --user is present, the hadoop_user specified will be used to run the tasks on the cluster. If unspecified, current user will be used (which should be valid in cluster). - Once hadoop security support is added, and if hadoop cluster is enabled with security, additional restrictions would apply via delegation tokens passed. +- The local directories used for spark will be the local directories configured for YARN (Hadoop Yarn config yarn.nodemanager.local-dirs). If the user specifies spark.local.dir, it will be ignored. -- cgit v1.2.3