diff options
| author | Lianhui Wang <lianhuiwang09@gmail.com> | 2016-04-19 19:48:03 -0700 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2016-04-19 19:48:03 -0700 |
| commit | 4514aebd1e807a665c270bfdc3f1127b3a1da898 (patch) | |
| tree | a12c92c3b577cf671694f50f47093dfab95401c1 /R/pkg/inst/tests/testthat | |
| parent | 3ae25f244bd471ef77002c703f2cc7ed6b524f11 (diff) | |
| download | spark-4514aebd1e807a665c270bfdc3f1127b3a1da898.tar.gz spark-4514aebd1e807a665c270bfdc3f1127b3a1da898.tar.bz2 spark-4514aebd1e807a665c270bfdc3f1127b3a1da898.zip | |
[SPARK-14705][YARN] support Multiple FileSystem for YARN STAGING DIR
## What changes were proposed in this pull request?
In SPARK-13063, It makes the SPARK YARN STAGING DIR as configurable. But it only support default FileSystem. If there are many clusters, It can be different FileSystem for different cluster in our spark.
## How was this patch tested?
I have tested it successfully with following commands:
MASTER=yarn-client ./bin/spark-shell --conf spark.yarn.stagingDir=hdfs:namenode2/temp
$SPARK_HOME/bin/spark-submit --conf spark.yarn.stagingDir=hdfs:namenode2/temp
cc tgravescs vanzin andrewor14
Author: Lianhui Wang <lianhuiwang09@gmail.com>
Closes #12473 from lianhuiwang/SPARK-14705.
Diffstat (limited to 'R/pkg/inst/tests/testthat')
0 files changed, 0 insertions, 0 deletions