diff options
| author | Kousuke Saruta <sarutak@oss.nttdata.co.jp> | 2015-05-15 11:37:34 +0100 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-05-15 11:37:34 +0100 |
| commit | c64ff8036cc6bc7c87743f4c751d7fe91c2e366a (patch) | |
| tree | 51f10802903deb378511a945c8db7624a5590615 /python | |
| parent | fdf5bba35d201fe0de3901b4d47262c485c76569 (diff) | |
| download | spark-c64ff8036cc6bc7c87743f4c751d7fe91c2e366a.tar.gz spark-c64ff8036cc6bc7c87743f4c751d7fe91c2e366a.tar.bz2 spark-c64ff8036cc6bc7c87743f4c751d7fe91c2e366a.zip | |
[SPARK-7503] [YARN] Resources in .sparkStaging directory can't be cleaned up on error
When we run applications on YARN with cluster mode, uploaded resources on .sparkStaging directory can't be cleaned up in case of failure of uploading local resources.
You can see this issue by running following command.
```
bin/spark-submit --master yarn --deploy-mode cluster --class <someClassName> <non-existing-jar>
```
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Closes #6026 from sarutak/delete-uploaded-resources-on-error and squashes the following commits:
caef9f4 [Kousuke Saruta] Fixed style
882f921 [Kousuke Saruta] Wrapped Client#submitApplication with try/catch blocks in order to delete resources on error
1786ca4 [Kousuke Saruta] Merge branch 'master' of https://github.com/apache/spark into delete-uploaded-resources-on-error
f61071b [Kousuke Saruta] Fixed cleanup problem
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions