diff options
| author | Thomas Graves <tgraves@apache.org> | 2014-01-10 15:34:15 -0600 |
|---|---|---|
| committer | Thomas Graves <tgraves@apache.org> | 2014-01-10 15:34:15 -0600 |
| commit | 7cef8435d7b6b43a33e8be684c769412186ad6ac (patch) | |
| tree | c1269c9c83abbb22262e98855e6d325d99c82ca2 /core | |
| parent | 7b58f116e5a028336e25a26daae5852c95e56340 (diff) | |
| parent | 9bdfbc0492f2d7408c250ae165763719cf290eeb (diff) | |
| download | spark-7cef8435d7b6b43a33e8be684c769412186ad6ac.tar.gz spark-7cef8435d7b6b43a33e8be684c769412186ad6ac.tar.bz2 spark-7cef8435d7b6b43a33e8be684c769412186ad6ac.zip | |
Merge pull request #371 from tgravescs/yarn_client_addjar_misc_fixes
Yarn client addjar and misc fixes
Fix the addJar functionality in yarn-client mode, add support for the other options supported in yarn-standalone mode, set the application type on yarn in hadoop 2.X, add documentation, change heartbeat interval to be same code as the yarn-standalone so it doesn't take so long to get containers and exit.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/SparkContext.scala | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala index 66c226e491..1647d904a2 100644 --- a/core/src/main/scala/org/apache/spark/SparkContext.scala +++ b/core/src/main/scala/org/apache/spark/SparkContext.scala @@ -677,10 +677,10 @@ class SparkContext( key = uri.getScheme match { // A JAR file which exists only on the driver node case null | "file" => - if (SparkHadoopUtil.get.isYarnMode()) { - // In order for this to work on yarn the user must specify the --addjars option to - // the client to upload the file into the distributed cache to make it show up in the - // current working directory. + if (SparkHadoopUtil.get.isYarnMode() && master == "yarn-standalone") { + // In order for this to work in yarn standalone mode the user must specify the + // --addjars option to the client to upload the file into the distributed cache + // of the AM to make it show up in the current working directory. val fileName = new Path(uri.getPath).getName() try { env.httpFileServer.addJar(new File(fileName)) |