diff options
| author | Thomas Graves <tgraves@prevailsail.corp.gq1.yahoo.com> | 2016-09-09 13:43:32 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-09-09 13:43:32 -0500 |
| commit | a3981c28c956a82ccf5b1c61d45b6bd252d4abed (patch) | |
| tree | 72e420d0b8111a5c33a3d57804aa32c76531f672 /bin/run-example | |
| parent | 7098a12945e71a159784836b75da855a603e1631 (diff) | |
| download | spark-a3981c28c956a82ccf5b1c61d45b6bd252d4abed.tar.gz spark-a3981c28c956a82ccf5b1c61d45b6bd252d4abed.tar.bz2 spark-a3981c28c956a82ccf5b1c61d45b6bd252d4abed.zip | |
[SPARK-17433] YarnShuffleService doesn't handle moving credentials levelDb
The secrets leveldb isn't being moved if you run spark shuffle services without yarn nm recovery on and then turn it on. This fixes that. I unfortunately missed this when I ported the patch from our internal branch 2 to master branch due to the changes for the recovery path. Note this only applies to master since it is the only place the yarn nm recovery dir is used.
Unit tests ran and tested on 8 node cluster. Fresh startup with NM recovery, fresh startup no nm recovery, switching between no nm recovery and recovery. Also tested running applications to make sure wasn't affected by rolling upgrade.
Author: Thomas Graves <tgraves@prevailsail.corp.gq1.yahoo.com>
Author: Tom Graves <tgraves@apache.org>
Closes #14999 from tgravescs/SPARK-17433.
Diffstat (limited to 'bin/run-example')
0 files changed, 0 insertions, 0 deletions