diff options
| author | Yanbo Liang <ybliang8@gmail.com> | 2017-03-21 21:50:54 -0700 |
|---|---|---|
| committer | Yanbo Liang <ybliang8@gmail.com> | 2017-03-21 21:50:54 -0700 |
| commit | 478fbc866fbfdb4439788583281863ecea14e8af (patch) | |
| tree | 5e2e46c53ec06403b06e086a983395a0bf4b8c32 /core | |
| parent | c1e87e384d1878308b42da80bb3d65be512aab55 (diff) | |
| download | spark-478fbc866fbfdb4439788583281863ecea14e8af.tar.gz spark-478fbc866fbfdb4439788583281863ecea14e8af.tar.bz2 spark-478fbc866fbfdb4439788583281863ecea14e8af.zip | |
[SPARK-19925][SPARKR] Fix SparkR spark.getSparkFiles fails when it was called on executors.
## What changes were proposed in this pull request?
SparkR ```spark.getSparkFiles``` fails when it was called on executors, see details at [SPARK-19925](https://issues.apache.org/jira/browse/SPARK-19925).
## How was this patch tested?
Add unit tests, and verify this fix at standalone and yarn cluster.
Author: Yanbo Liang <ybliang8@gmail.com>
Closes #17274 from yanboliang/spark-19925.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/api/r/RRunner.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/api/r/RRunner.scala b/core/src/main/scala/org/apache/spark/api/r/RRunner.scala index 29e21b3b1a..8811839200 100644 --- a/core/src/main/scala/org/apache/spark/api/r/RRunner.scala +++ b/core/src/main/scala/org/apache/spark/api/r/RRunner.scala @@ -347,6 +347,8 @@ private[r] object RRunner { pb.environment().put("SPARKR_RLIBDIR", rLibDir.mkString(",")) pb.environment().put("SPARKR_WORKER_PORT", port.toString) pb.environment().put("SPARKR_BACKEND_CONNECTION_TIMEOUT", rConnectionTimeout.toString) + pb.environment().put("SPARKR_SPARKFILES_ROOT_DIR", SparkFiles.getRootDirectory()) + pb.environment().put("SPARKR_IS_RUNNING_ON_WORKER", "TRUE") pb.redirectErrorStream(true) // redirect stderr into stdout val proc = pb.start() val errThread = startStdoutThread(proc) |