Adding in the --addJars option to make SparkContext.addJar work on yarn and cleanup

the classpaths

1 files changed, 2 insertions, 0 deletions

diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index beaae69aa2..a807ec603d 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -53,6 +53,7 @@ The command to launch the YARN Client is as follows:
       --worker-memory <MEMORY_PER_WORKER> \
       --worker-cores <CORES_PER_WORKER> \
       --queue <queue_name> \
+      --addJars <any_local_files_used_in_SparkContext.addJar> \
       --files <files_for_distributed_cache> \
       --archives <archives_for_distributed_cache>
 
@@ -88,3 +89,4 @@ The above starts a YARN Client programs which periodically polls the Application
 - We do not requesting container resources based on the number of cores. Thus the numbers of cores given via command line arguments cannot be guaranteed.
 - The local directories used for spark will be the local directories configured for YARN (Hadoop Yarn config yarn.nodemanager.local-dirs). If the user specifies spark.local.dir, it will be ignored.
 - The --files and --archives options support specifying file names with the # similar to Hadoop. For example you can specify: --files localtest.txt#appSees.txt and this will upload the file you have locally named localtest.txt into HDFS but this will be linked to by the name appSees.txt and your application should use the name as appSees.txt to reference it when running on YARN.
+- The --addJars option allows the SparkContext.addJar function to work if you are using it with local files. It does not need to be used if you are using it with HDFS, HTTP, HTTPS, or FTP files.