diff options
| author | jerryshao <sshao@hortonworks.com> | 2017-01-17 09:30:56 -0600 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2017-01-17 09:30:56 -0600 |
| commit | b79cc7ceb439b3d4e0009963ede3416e3241e562 (patch) | |
| tree | 7d28019c5144cbb5094edf433fb70e59ffc121e6 /core | |
| parent | 6c00c069e3c3f5904abd122cea1d56683031cca0 (diff) | |
| download | spark-b79cc7ceb439b3d4e0009963ede3416e3241e562.tar.gz spark-b79cc7ceb439b3d4e0009963ede3416e3241e562.tar.bz2 spark-b79cc7ceb439b3d4e0009963ede3416e3241e562.zip | |
[SPARK-19179][YARN] Change spark.yarn.access.namenodes config and update docs
## What changes were proposed in this pull request?
`spark.yarn.access.namenodes` configuration cannot actually reflects the usage of it, inside the code it is the Hadoop filesystems we get tokens, not NNs. So here propose to update the name of this configuration, also change the related code and doc.
## How was this patch tested?
Local verification.
Author: jerryshao <sshao@hortonworks.com>
Closes #16560 from jerryshao/SPARK-19179.
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/SparkConf.scala | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkConf.scala b/core/src/main/scala/org/apache/spark/SparkConf.scala index d78b9f1b29..601d24191e 100644 --- a/core/src/main/scala/org/apache/spark/SparkConf.scala +++ b/core/src/main/scala/org/apache/spark/SparkConf.scala @@ -699,8 +699,10 @@ private[spark] object SparkConf extends Logging { "spark.rpc.message.maxSize" -> Seq( AlternateConfig("spark.akka.frameSize", "1.6")), "spark.yarn.jars" -> Seq( - AlternateConfig("spark.yarn.jar", "2.0")) - ) + AlternateConfig("spark.yarn.jar", "2.0")), + "spark.yarn.access.hadoopFileSystems" -> Seq( + AlternateConfig("spark.yarn.access.namenodes", "2.2")) + ) /** * A view of `configsWithAlternatives` that makes it more efficient to look up deprecated |