diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2016-08-17 11:12:21 -0700 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2016-08-17 11:12:21 -0700 |
| commit | e3fec51fa1ed161789ab7aa32ed36efe357b5d31 (patch) | |
| tree | 963eced70149018b74120f532e97484c0d69226e /R | |
| parent | 928ca1c6d12b23d84f9b6205e22d2e756311f072 (diff) | |
| download | spark-e3fec51fa1ed161789ab7aa32ed36efe357b5d31.tar.gz spark-e3fec51fa1ed161789ab7aa32ed36efe357b5d31.tar.bz2 spark-e3fec51fa1ed161789ab7aa32ed36efe357b5d31.zip | |
[SPARK-16930][YARN] Fix a couple of races in cluster app initialization.
There are two narrow races that could cause the ApplicationMaster to miss
when the user application instantiates the SparkContext, which could cause
app failures when nothing was wrong with the app. It was also possible for
a failing application to get stuck in the loop that waits for the context
for a long time, instead of failing quickly.
The change uses a promise to track the SparkContext instance, which gets
rid of the races and allows for some simplification of the code.
Tested with existing unit tests, and a new one being added to test the
timeout code.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #14542 from vanzin/SPARK-16930.
Diffstat (limited to 'R')
0 files changed, 0 insertions, 0 deletions