diff options
| author | Sandy Ryza <sandy@cloudera.com> | 2014-12-09 11:02:43 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-12-09 11:02:43 -0800 |
| commit | 912563aa3553afc0871d5b5858f533aa39cb99e5 (patch) | |
| tree | 092241ac4c78deef8053f5095bc9680b3c8532cd /docs/running-on-yarn.md | |
| parent | 383c5555c9f26c080bc9e3a463aab21dd5b3797f (diff) | |
| download | spark-912563aa3553afc0871d5b5858f533aa39cb99e5.tar.gz spark-912563aa3553afc0871d5b5858f533aa39cb99e5.tar.bz2 spark-912563aa3553afc0871d5b5858f533aa39cb99e5.zip | |
SPARK-4338. [YARN] Ditch yarn-alpha.
Sorry if this is a little premature with 1.2 still not out the door, but it will make other work like SPARK-4136 and SPARK-2089 a lot easier.
Author: Sandy Ryza <sandy@cloudera.com>
Closes #3215 from sryza/sandy-spark-4338 and squashes the following commits:
1c5ac08 [Sandy Ryza] Update building Spark docs and remove unnecessary newline
9c1421c [Sandy Ryza] SPARK-4338. Ditch yarn-alpha.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 16897dbb65..62b317129b 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -216,7 +216,7 @@ If you need a reference to the proper location to put log files in the YARN so t # Important notes -- Before Hadoop 2.2, YARN does not support cores in container resource requests. Thus, when running against an earlier version, the numbers of cores given via command line arguments cannot be passed to YARN. Whether core requests are honored in scheduling decisions depends on which scheduler is in use and how it is configured. +- Whether core requests are honored in scheduling decisions depends on which scheduler is in use and how it is configured. - The local directories used by Spark executors will be the local directories configured for YARN (Hadoop YARN config `yarn.nodemanager.local-dirs`). If the user specifies `spark.local.dir`, it will be ignored. - The `--files` and `--archives` options support specifying file names with the # similar to Hadoop. For example you can specify: `--files localtest.txt#appSees.txt` and this will upload the file you have locally named localtest.txt into HDFS but this will be linked to by the name `appSees.txt`, and your application should use the name as `appSees.txt` to reference it when running on YARN. - The `--jars` option allows the `SparkContext.addJar` function to work if you are using it with local files and running in `yarn-cluster` mode. It does not need to be used if you are using it with HDFS, HTTP, HTTPS, or FTP files. |