diff options
| author | Burak Yavuz <brkyvz@gmail.com> | 2016-11-08 15:08:09 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-11-08 15:08:09 -0800 |
| commit | 6f7ecb0f2975d24a71e4240cf623f5bd8992bbeb (patch) | |
| tree | a018e0871a7757de11c8b1e943f8d0749d509675 /sql/hive | |
| parent | b6de0c98c70960a97b07615b0b08fbd8f900fbe7 (diff) | |
| download | spark-6f7ecb0f2975d24a71e4240cf623f5bd8992bbeb.tar.gz spark-6f7ecb0f2975d24a71e4240cf623f5bd8992bbeb.tar.bz2 spark-6f7ecb0f2975d24a71e4240cf623f5bd8992bbeb.zip | |
[SPARK-18342] Make rename failures fatal in HDFSBackedStateStore
## What changes were proposed in this pull request?
If the rename operation in the state store fails (`fs.rename` returns `false`), the StateStore should throw an exception and have the task retry. Currently if renames fail, nothing happens during execution immediately. However, you will observe that snapshot operations will fail, and then any attempt at recovery (executor failure / checkpoint recovery) also fails.
## How was this patch tested?
Unit test
Author: Burak Yavuz <brkyvz@gmail.com>
Closes #15804 from brkyvz/rename-state.
Diffstat (limited to 'sql/hive')
0 files changed, 0 insertions, 0 deletions