diff options
| author | Mridul Muralidharan <mridul@gmail.com> | 2013-05-11 11:12:22 +0530 |
|---|---|---|
| committer | Mridul Muralidharan <mridul@gmail.com> | 2013-05-11 11:12:22 +0530 |
| commit | ee37612bc95e8486fa328908005293585912db71 (patch) | |
| tree | 8a9df46b29cd5d808f93a24eebdd09223e7e2e70 /docs/running-on-yarn.md | |
| parent | b05c9d22d70333924b988b2dfa359ce3e11f7c9d (diff) | |
| download | spark-ee37612bc95e8486fa328908005293585912db71.tar.gz spark-ee37612bc95e8486fa328908005293585912db71.tar.bz2 spark-ee37612bc95e8486fa328908005293585912db71.zip | |
1) Add support for HADOOP_CONF_DIR (and/or YARN_CONF_DIR - use either) : which is used to specify the client side configuration directory : which needs to be part of the CLASSPATH.
2) Move from var+=".." to var="$var.." : the former does not work on older bash shells unfortunately.
Diffstat (limited to 'docs/running-on-yarn.md')
| -rw-r--r-- | docs/running-on-yarn.md | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 26424bbe52..c8cf8ffc35 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -30,6 +30,9 @@ If you want to test out the YARN deployment mode, you can use the current Spark # Launching Spark on YARN +Ensure that HADOOP_CONF_DIR or YARN_CONF_DIR points to the directory which contains the (client side) configuration files for the hadoop cluster. +This would be used to connect to the cluster, write to the dfs and submit jobs to the resource manager. + The command to launch the YARN Client is as follows: SPARK_JAR=<SPARK_YAR_FILE> ./run spark.deploy.yarn.Client \ |