diff options
| author | Cheng Lian <lian@databricks.com> | 2015-06-06 17:23:12 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2015-06-06 17:23:12 +0800 |
| commit | 16fc49617e1dfcbe9122b224f7f63b7bfddb36ce (patch) | |
| tree | 8c342d1910833f47e65a1acfaa7dd6b0b1f36e13 /python/run-tests | |
| parent | a8077e5cfc48bdb9f0641d62fe6c01cc8c4f1694 (diff) | |
| download | spark-16fc49617e1dfcbe9122b224f7f63b7bfddb36ce.tar.gz spark-16fc49617e1dfcbe9122b224f7f63b7bfddb36ce.tar.bz2 spark-16fc49617e1dfcbe9122b224f7f63b7bfddb36ce.zip | |
[SPARK-8079] [SQL] Makes InsertIntoHadoopFsRelation job/task abortion more robust
As described in SPARK-8079, when writing a DataFrame to a `HadoopFsRelation`, if `HadoopFsRelation.prepareForWriteJob` throws exception, an unexpected NPE will be thrown during job abortion. (This issue doesn't bring much damage since the job is failing anyway.)
This PR makes the job/task abortion logic in `InsertIntoHadoopFsRelation` more robust to avoid such confusing exceptions.
Author: Cheng Lian <lian@databricks.com>
Closes #6612 from liancheng/spark-8079 and squashes the following commits:
87cd81e [Cheng Lian] Addresses @rxin's comment
1864c75 [Cheng Lian] Addresses review comments
9e6dbb3 [Cheng Lian] Makes InsertIntoHadoopFsRelation job/task abortion more robust
Diffstat (limited to 'python/run-tests')
0 files changed, 0 insertions, 0 deletions