diff options
| author | zsxwing <zsxwing@gmail.com> | 2014-10-22 15:04:41 -0700 |
|---|---|---|
| committer | Andrew Or <andrewor14@gmail.com> | 2014-10-22 15:04:41 -0700 |
| commit | 137d94235383cc49ccf8a7bb7f314f578aa1dede (patch) | |
| tree | 650feb18ef7afd0a2bbe3d03bcacc834f8428647 /pom.xml | |
| parent | 813effc701fc27121c6f23ab32882932016fdbe0 (diff) | |
| download | spark-137d94235383cc49ccf8a7bb7f314f578aa1dede.tar.gz spark-137d94235383cc49ccf8a7bb7f314f578aa1dede.tar.bz2 spark-137d94235383cc49ccf8a7bb7f314f578aa1dede.zip | |
[SPARK-3877][YARN] Throw an exception when application is not successful so that the exit code wil be set to 1
When an yarn application fails (yarn-cluster mode), the exit code of spark-submit is still 0. It's hard for people to write some automatic scripts to run spark jobs in yarn because the failure can not be detected in these scripts.
This PR added a status checking after `monitorApplication`. If an application is not successful, `run()` will throw an `SparkException`, so that Client.scala will exit with code 1. Therefore, people can use the exit code of `spark-submit` to write some automatic scripts.
Author: zsxwing <zsxwing@gmail.com>
Closes #2732 from zsxwing/SPARK-3877 and squashes the following commits:
1f89fa5 [zsxwing] Fix the unit test
a0498e1 [zsxwing] Update the docs and the error message
e1cb9ef [zsxwing] Fix the hacky way of calling Client
ff16fec [zsxwing] Remove System.exit in Client.scala and add a test
6a2c103 [zsxwing] [SPARK-3877] Throw an exception when application is not successful so that the exit code wil be set to 1
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions