diff options
| author | n.fraison <n.fraison@criteo.com> | 2016-11-25 09:45:51 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-11-25 09:45:51 +0000 |
| commit | f42db0c0c1434bfcccaa70d0db55e16c4396af04 (patch) | |
| tree | 3b1513a4a0e8163a7b9590884a1409f2c726f1a8 /core/src/main/scala/org/apache | |
| parent | f58a8aa20106ea36386db79a8a66f529a8da75c9 (diff) | |
| download | spark-f42db0c0c1434bfcccaa70d0db55e16c4396af04.tar.gz spark-f42db0c0c1434bfcccaa70d0db55e16c4396af04.tar.bz2 spark-f42db0c0c1434bfcccaa70d0db55e16c4396af04.zip | |
[SPARK-18119][SPARK-CORE] Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server
## What changes were proposed in this pull request?
Instead of using the setSafeMode method that check the first namenode used the one which permitts to check only for active NNs
## How was this patch tested?
manual tests
Please review https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before opening a pull request.
This commit is contributed by Criteo SA under the Apache v2 licence.
Author: n.fraison <n.fraison@criteo.com>
Closes #15648 from ashangit/SPARK-18119.
Diffstat (limited to 'core/src/main/scala/org/apache')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala index ca38a47639..8ef69b142c 100644 --- a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala +++ b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala @@ -663,9 +663,9 @@ private[history] class FsHistoryProvider(conf: SparkConf, clock: Clock) false } - // For testing. private[history] def isFsInSafeMode(dfs: DistributedFileSystem): Boolean = { - dfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_GET) + /* true to check only for Active NNs status */ + dfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_GET, true) } /** |