diff options
author | WangTaoTheTonic <wangtao111@huawei.com> | 2015-05-01 18:38:20 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-05-01 18:38:20 -0700 |
commit | b4b43df8a338a30c0eadcf10cbe3ba203dc3f861 (patch) | |
tree | 1cce6e72d9c907398ef6ffb89986ae03c8ba729b /docker | |
parent | 2022193412e832393a29b94609841c3ffe8e3d66 (diff) | |
download | spark-b4b43df8a338a30c0eadcf10cbe3ba203dc3f861.tar.gz spark-b4b43df8a338a30c0eadcf10cbe3ba203dc3f861.tar.bz2 spark-b4b43df8a338a30c0eadcf10cbe3ba203dc3f861.zip |
[SPARK-6443] [SPARK SUBMIT] Could not submit app in standalone cluster mode when HA is enabled
**3/26 update:**
* Akka-based:
Use an array of `ActorSelection` to represent multiple master. Add an `activeMasterActor` for query status of driver. And will add lost masters( including the standby one) to `lostMasters`.
When size of `lostMasters` equals or greater than # of all masters, we should give an error that all masters are not avalible.
* Rest-based:
When all masters are not available(throw an exception), we use akka gateway to submit apps.
I have tested simply on standalone HA cluster(with two masters alive and one alive/one dead), it worked.
There might remains some issues on style or message print, but we can check the solution then fix them together.
/cc srowen andrewor14
Author: WangTaoTheTonic <wangtao111@huawei.com>
Closes #5116 from WangTaoTheTonic/SPARK-6443 and squashes the following commits:
2a28aab [WangTaoTheTonic] based the newest change https://github.com/apache/spark/pull/5144
76fd411 [WangTaoTheTonic] rebase
f4f972b [WangTaoTheTonic] rebase...again
a41de0b [WangTaoTheTonic] rebase
220cb3c [WangTaoTheTonic] move connect exception inside
35119a0 [WangTaoTheTonic] style and compile issues
9d636be [WangTaoTheTonic] per Andrew's comments
979760c [WangTaoTheTonic] rebase
e4f4ece [WangTaoTheTonic] fix failed test
5d23958 [WangTaoTheTonic] refact some duplicated code, style and comments
7a881b3 [WangTaoTheTonic] when one of masters is gone, we still can submit
2b011c9 [WangTaoTheTonic] fix broken tests
60d97a4 [WangTaoTheTonic] rebase
fa1fa80 [WangTaoTheTonic] submit app to HA cluster in standalone cluster mode
Diffstat (limited to 'docker')
0 files changed, 0 insertions, 0 deletions