From 5a92dc76ab431d73275a2bdfbc2c0a8ceb0d75d1 Mon Sep 17 00:00:00 2001 From: Nicholas Chammas Date: Mon, 5 Dec 2016 12:57:41 -0800 Subject: [DOCS][MINOR] Update location of Spark YARN shuffle jar Looking at the distributions provided on spark.apache.org, I see that the Spark YARN shuffle jar is under `yarn/` and not `lib/`. This change is so minor I'm not sure it needs a JIRA. But let me know if so and I'll create one. Author: Nicholas Chammas Closes #16130 from nchammas/yarn-doc-fix. --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/running-on-yarn.md') diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 4d1fafc07b..d4144c86e9 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -518,7 +518,7 @@ instructions: pre-packaged distribution. 1. Locate the `spark--yarn-shuffle.jar`. This should be under `$SPARK_HOME/common/network-yarn/target/scala-` if you are building Spark yourself, and under -`lib` if you are using a distribution. +`yarn` if you are using a distribution. 1. Add this jar to the classpath of all `NodeManager`s in your cluster. 1. In the `yarn-site.xml` on each node, add `spark_shuffle` to `yarn.nodemanager.aux-services`, then set `yarn.nodemanager.aux-services.spark_shuffle.class` to -- cgit v1.2.3