diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2014-04-17 10:29:38 -0500 |
|---|---|---|
| committer | Thomas Graves <tgraves@apache.org> | 2014-04-17 10:29:38 -0500 |
| commit | 69047506bf97e6e37e4079c87cb0327d3760ac41 (patch) | |
| tree | f1ffe2efe61f0eb5c379aab43f35c751164baf2b /yarn/stable | |
| parent | bb76eae1b50e4bf18360220110f7d0a4bee672ec (diff) | |
| download | spark-69047506bf97e6e37e4079c87cb0327d3760ac41.tar.gz spark-69047506bf97e6e37e4079c87cb0327d3760ac41.tar.bz2 spark-69047506bf97e6e37e4079c87cb0327d3760ac41.zip | |
[SPARK-1395] Allow "local:" URIs to work on Yarn.
This only works for the three paths defined in the environment
(SPARK_JAR, SPARK_YARN_APP_JAR and SPARK_LOG4J_CONF).
Tested by running SparkPi with local: and file: URIs against Yarn cluster (no "upload" shows up in logs in the local case).
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #303 from vanzin/yarn-local and squashes the following commits:
82219c1 [Marcelo Vanzin] [SPARK-1395] Allow "local:" URIs to work on Yarn.
Diffstat (limited to 'yarn/stable')
| -rw-r--r-- | yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala index 81d9d1b5c9..117b33f466 100644 --- a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala +++ b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala @@ -79,7 +79,7 @@ class ExecutorRunnable( ctx.setTokens(ByteBuffer.wrap(dob.getData())) val commands = prepareCommand(masterAddress, slaveId, hostname, executorMemory, executorCores, - localResources.contains(ClientBase.LOG4J_PROP)) + localResources) logInfo("Setting up executor with commands: " + commands) ctx.setCommands(commands) |