diff options
author | Marcelo Vanzin <vanzin@cloudera.com> | 2014-04-17 10:29:38 -0500 |
---|---|---|
committer | Thomas Graves <tgraves@apache.org> | 2014-04-17 10:33:15 -0500 |
commit | a83a794f1accc616cfccde78af44c5cbf066c647 (patch) | |
tree | 8a209a95e511e4a4e0b078640a9915d1824d2c0c /yarn/stable | |
parent | b3ad707c4411c3860691c9eb802fec425bf29e85 (diff) | |
download | spark-a83a794f1accc616cfccde78af44c5cbf066c647.tar.gz spark-a83a794f1accc616cfccde78af44c5cbf066c647.tar.bz2 spark-a83a794f1accc616cfccde78af44c5cbf066c647.zip |
[SPARK-1395] Allow "local:" URIs to work on Yarn.
This only works for the three paths defined in the environment
(SPARK_JAR, SPARK_YARN_APP_JAR and SPARK_LOG4J_CONF).
Tested by running SparkPi with local: and file: URIs against Yarn cluster (no "upload" shows up in logs in the local case).
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #303 from vanzin/yarn-local and squashes the following commits:
82219c1 [Marcelo Vanzin] [SPARK-1395] Allow "local:" URIs to work on Yarn.
(cherry picked from commit 69047506bf97e6e37e4079c87cb0327d3760ac41)
Signed-off-by: Thomas Graves <tgraves@apache.org>
Diffstat (limited to 'yarn/stable')
-rw-r--r-- | yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala index 81d9d1b5c9..117b33f466 100644 --- a/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala +++ b/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala @@ -79,7 +79,7 @@ class ExecutorRunnable( ctx.setTokens(ByteBuffer.wrap(dob.getData())) val commands = prepareCommand(masterAddress, slaveId, hostname, executorMemory, executorCores, - localResources.contains(ClientBase.LOG4J_PROP)) + localResources) logInfo("Setting up executor with commands: " + commands) ctx.setCommands(commands) |